The messaging service
The messaging service is the durable event backbone of a cell. It carries
durable publish/subscribe, runs streaming SQL and pipelines over the message
bus, bridges to external transports, and replicates messages across cells. Every
message carries a workspace scope, so the same governance that covers data covers
events.
Feeds: Data Platform.
It persists through FluxDB into the
messages logical database, uses the same hot/warm/cold tiering onto the storage
service's object store, converges with the same conflict-free primitives as the
data plane, and shares the stream connector layer with data ingestion rather
than duplicating it. The separation into its own service is a runtime
decomposition, not a separate product.
What it owns
- Durable pub/sub: persistent topics with at-least-once delivery.
- Streaming SQL: queries that run continuously over the message stream.
- Pipelines: quality-checked, lineage-tracked message pipelines.
- Transports: bridges to external message systems.
- Cross-cell replication: copying and cloning topics between cells.
Sub-modules (public names)
| Sub-module | What it does |
|---|---|
| Messaging & Streaming | The durable pub/sub engine and its console surface |
| Routing | Topic routing and subscription matching |
| Transport bridges | Bridges to external message systems |
| Streaming | Streaming SQL over the bus |
| Pipelines | Quality-checked, lineage-tracked pipelines |
| Replication | Cross-cell topic replication and cloning |
| Events | Platform event fan-out |
| Feed & Publishing | The feed and publishing layer over the bus |
| Cloud transports | Connectors to cloud messaging services |
APIs and surfaces
The messaging service exposes its pub/sub and streaming surfaces over HTTP and
gRPC: a loopback clear port for intra-cell callers and a mutual-TLS port
advertised on the fabric for cross-cell replication (bound on all interfaces,
advertised as the cell FQDN). External clients reach messaging only through the
node front door.
Two outbound transport bridges are configured, an AMQP bridge and an MQTT bridge, for interoperating with external brokers.
Both the AMQP and MQTT transport bridges ship disabled by default and with no
broker host configured. They are outbound client connectors, not inbound
listeners, the messaging service binds no AMQP or MQTT port of its own. Point
them at a real broker and enable them to use them. See the
Ports & protocols reference.
Workers
Background workers cover topic compaction, delivery retry, pipeline execution and quality checks, lineage tracking, and cross-cell replication catch-up. Each runs per cell.
The publish path
Figure: the publish path. A durable topic (filled, core) fans to subscribers, streaming SQL, and cross-cell replication (outlined, mutual TLS). External transport bridges are drawn amber / dotted because they are held off by default and must be opted into. Ceiling: delivery is at-least-once; the diagram shows the message path, not the exactly-once guarantees of any downstream sink.
Table equivalent
| From | Relationship | To | State |
|---|---|---|---|
| Publisher | writes to | Durable topic | workspace-scoped |
| Durable topic | fans to | Subscribers (at-least-once) | neutral |
| Durable topic | feeds | Streaming SQL / pipelines | neutral |
| Durable topic | replicates via mTLS to | Peer cells | edge (cross-cell) |
| Durable topic | optionally bridges to | External broker | off by default |
In the deployment matrix
messaging runs identically everywhere. Single-cell, replication listeners are
idle and topics are local. Multi-cell, topics replicate and clone across cells
over mutual TLS. Sovereign / air-gapped, the bus is fully functional locally and
a peer that cannot be reached is an absent edge; external transport bridges stay
off unless an operator enables one against a reachable broker.
Related topics
- Data Platform, the area messaging serves.
- Replication, DR & federation, cross-cell replication.
- Ports & protocols reference.