Skip to main content

Connectors and ingestion

Most platforms count connectors by counting logos. This page counts them by counting drivers that run, and separates those from catalog content that describes an integration without implementing its protocol.

The short answer​

FamilyWhat it isHow many
Streaming and real-timeConcrete protocol drivers with poll, health and metrics, driven by a live worker106
Federated providersRead a source in place without copying it18 classes across 33 aliases
Vertical ETLIndustry-specific pipelines16
Security ingestFeeds into the event lake8
Business intelligenceOutbound, to the tools your analysts already use6
Connector templatesGoverned integration templates, not drivers. See below327

The number to hold onto is the first one. A hundred and six real-time connectors is the part of this that no amount of catalog content substitutes for.

Streaming and real-time​

These are the drivers. Each is a module with a poll or start loop, configuration, health, metrics and a worker tick, and each is registered in a factory that is verified to have no orphans.

What is unusual is not the count, it is the spread. Most data platforms cover the cloud and SaaS columns well and stop.

DomainProtocols
Generic transportsREST polling, WebSocket, server-sent events, webhook, MQTT, AMQP, syslog, database change data capture, file watch, Kafka, gRPC streaming
Cloud messagingSQS, SNS, Kinesis, EventBridge, Service Bus, Event Hubs, Event Grid, Pub/Sub and its push variant, Cloud Tasks
Industrial and operational technologyOPC UA, Modbus, DNP3, IEC 61850, BACnet, PROFINET, EtherCAT, CAN bus, MTConnect, and historian feeds
Tactical and defenseCursor on Target, JREAP, Link 16, VMF, blue-force tracking, MAVLink, KLV full-motion video, NITF, ASTERIX, ACARS, STANAG 4607, orbital element sets, CCSDS, tactical radio, direction finding, software-defined radio, spectrum
Sensors and geospatialAIS, ADS-B, sonar, APRS, LoRa, mesh telemetry, NAVTEX, NMEA GPS, geofence, METAR, chemical and radiological, SEG-Y, WITSML
HealthcareFHIR, HL7 minimal lower layer protocol, DICOM, IEEE 11073, C-CDA, NCPDP
FinanceFIX, ISO 20022, market data, SWIFT, open banking, market terminals
SecuritySplunk, Microsoft Sentinel, CrowdStrike, STIX/TAXII, NetFlow, Zeek, Suricata, Snort, Palo Alto, Fortinet, Qualys, Tenable, vulnerability feeds, OCSF, Wazuh, Elastic, Carbon Black, packet capture, DNS
Documents and verticalsMailbox archives, building information models, legal billing, education data, EDI, broadcast cueing, network device streams
WebFeeds and the crawler below

Two properties hold across all of them: a pull connector is driven by a poller and a push connector by a runner with exponential backoff, and every push ingress is verified per connector rather than accepting whatever arrives.

Federated providers: read it where it is​

A federated provider connects to a system and reads it in place. Nothing is copied, and the source stays authoritative.

Object stores (S3, Azure, Google Cloud Storage, and the Parquet, Delta and Iceberg layouts over each), relational databases (PostgreSQL, MySQL, SQL Server), warehouses and lakehouses (BigQuery, Databricks and its catalog, Snowflake's catalog, Microsoft Fabric), open sharing protocols, and content platforms (SharePoint, Power BI, OneDrive, Google Drive, Dropbox).

Live connections are cached per connection with a health check on first use, and an unhealthy provider raises rather than handing back a dead handle.

The web crawler​

The platform can crawl and ingest external websites, and the interesting part is how tightly it is bounded.

ControlBehaviour
ScopeSeed URLs, a maximum depth, and a maximum total unique URLs. Both bounds apply
FrontierBreadth-first, deduplicated by canonical URL, so a site's own link graph cannot loop it
RobotsHonoured, with no override and no evasion path. A disallow means stop
PolitenessA distributed per-workspace, per-host token bucket, so courtesy holds across cells and one workspace cannot exhaust another's budget for a domain
Crawl delayTaken from the site's own directive where present, with a courtesy default
BackpressureThe crawler pauses when downstream extraction backs up, with hysteresis so it does not oscillate
AccountingAn append-only, hash-chained fetch ledger per workspace: pages fetched, bytes, egress, rate-limit hits, robots skips, per-domain attribution

Robots checking happens before a rate token is spent, so a disallowed URL costs nothing.

A fetched page becomes a governed record carrying its title, text and links, and enters the same unstructured funnel as any other document. Provenance is stamped at ingest: web content is labelled at a sensitivity that escalates but never downgrades, and an external-trust source is refused entry to a clean room, fail-closed.

Right to erasure is implemented as a tombstone-and-purge across every store that holds the content, and it fails closed if any store has no wired deleter.

The template catalog, and what it is actually for​

There are 327 named connector templates. It is worth being blunt about what they are, because the number invites the wrong conclusion.

A template is not a driver. It carries no endpoint and no protocol code. It declares an integration's governance envelope: its category, its authentication method, the scope its credential lives at, and a tool allow list that acts as a ceiling on what the integration can ever expose.

A template becomes a working integration when an operator clones it, binds an endpoint and a workspace secret, and the platform discovers the remote's tools and projects them into the skills catalog. The remote has to speak the Model Context Protocol.

So what is the value?

  • The ceiling is declared and reviewed before anyone connects anything. When a template is cloned and bound, the allow list caps what tools can ever be projected from that remote, whatever the remote offers.
  • The sovereign variants are the clearest case. A hundred of the entries are government and regulated-partition variants. Those differ from their commercial counterparts in compliance posture and partition, not in protocol, and that distinction is exactly the kind of thing that belongs in reviewed catalog content rather than being rediscovered per deployment. They also explicitly refuse to assert an authorization the platform does not hold.
  • Credentials are references, never values. The catalog is checked for secret-shaped strings, and a clone deliberately drops the credential binding.
  • It is a discovery surface. Browsing what is available, by category, without reading source.
How to read the 327

Treat it as a governed template gallery for protocol-compatible integration, not as a count of working connectors. If a specific integration matters to you, the questions are whether the remote speaks the protocol and who runs it, not whether a template exists.

The number that answers "what can this platform ingest today" is the 106 streaming connectors and the 18 federated providers, not this one.

The other families​

Vertical ETL pipelines for sixteen industries: healthcare, insurance, finance, manufacturing, retail, energy, logistics, telecommunications, agriculture, government, education, media, real estate, oil and gas, cyber insurance and legal. Each maps a domain's own format into the platform's dimensional model.

Security ingest carries eight concrete feeds into the event lake, each normalising into an open event schema with dead-lettering at the normalise and ingest stages rather than dropping what it cannot parse.

Business intelligence is the one outbound family: six adapters, each pinned to its tool's real authentication flow, with the endpoint and credential always taken from a stored connection rather than from the caller.

Cloud enumeration reads a cloud account's inventory for Discovery and Mapping. AWS is available today; the other partitions are on the roadmap and register as explicitly pending rather than silently returning nothing.

What holds across every family​

A credential is a reference. Connections store a secret identifier, not a secret. Resolution happens in the service that owns the data path, and the resolved value never crosses back to the caller.

Unavailable is not empty. A connector family with no backend on this cell raises a named error. It never reports that it looked and found nothing, because a clean report on an estate that was never scanned is the most expensive kind of wrong answer.

Ingest is governed at entry. Provenance and sensitivity are stamped where data lands, escalate rather than downgrade, and an external-trust source cannot enter a clean room.