Multiple node egress per org
An organization is rarely one node. It can present multiple public front doors several nodes, each with its own front-door URL, and it must decide, for each kind of outbound traffic, which single cell opens the socket to the outside. This page covers both halves of that multi-node picture: the inbound side (multiple front doors, resolved deny-closed) and the outbound side (governed egress, where exactly one cell per org and traffic class is elected to own the connection).
The two halves are separate mechanisms and it helps to keep them apart: front-door / node-gateway resolution is a directory concern (which URL reaches which node's face), handled by the directory router; egress ownership is a lease-and-election concern (which cell dials out), living in the node's governed-egress layer.
The topology: one org, many front doors, one elected exit
Figure: an org's multi-node egress topology. Each node publishes its own
front-door URL, which resolves to that node's public-facing group (the
inbound side, many front doors per org). For outbound traffic, one cell (here
Cell B1, green) is the elected egress owner for a given org and traffic
class; every other cell (even on another node) forwards to the holder rather
than opening its own socket, so the org presents one governed exit per traffic
class. Ceiling: ownership is scoped per (org, traffic class), so different
traffic classes can have different owners on different nodes; the figure shows
one class. Front doors are unique (one URL maps to exactly one node) and a
peer's forward is accepted only by the current lease holder; a non-holder
refuses it.
Table equivalent
| Element | Role | Multiplicity |
|---|---|---|
| Organization | Owns the nodes and the traffic | one |
| Node front door (URL → public-facing group) | Inbound address of a node | many per org (one URL per node) |
| Cell | A candidate egress owner | many per node |
| Egress owner (lease holder) | Outbound: opens the one socket | exactly one per (org, traffic class) |
| Peer cell | Forwards its outbound bundle to the holder | the rest |
| External destination | The outbound target | per traffic class |
Inbound: multiple front doors, one URL to one node
A node's front door is its public-facing service group, and a node publishes it by setting a front-door URL on one of its gateway rows. An org can publish many (one per node) and the mapping is kept unambiguous by a uniqueness constraint: one active front-door URL maps to exactly one node. Resolution is deny-closed (an unmapped, ambiguous, or corrupt URL refuses rather than guesses), and it is the same resolver on the core and the edge, pinned by a cross-language fixture. The full resolve cascade, normalization, exact-host match, and the four refusal branches, is documented in the directory router.
A node gateway row carries more than the URL: the service name and protocol, host, port, TLS and mutual-TLS posture, topology (region, availability zone, locality), ranking (priority, cost tier, weight), and health (round-trip time, healthy flag, last-seen). The front-door URL is operator-set and deliberately survives the automated gateway sync that refreshes the health and topology fields.
Outbound: electing the cell that opens the socket
For outbound traffic, the platform does not let every cell dial out
independently. Governed egress elects one owner per (org, traffic class),
and peers forward to it. Traffic is split into classes so that different streams
can have different owners:
| Traffic class (public name) | What it carries |
|---|---|
| Proactive Support | Support telemetry envelopes (Proactive Support) |
| Metering | Usage/billing reporting (always-on plane) |
| License | License checks and refresh |
| Capability coordination | Cross-cell capability/grant coordination |
| Data plane | Governed data movement |
| Model inference | Brokered model/LLM calls |
| SIEM forwarding | Security event forwarding |
| Federation | Cross-org federated traffic |
Each class is a separate lease (its scope id is the org plus the class), so one node can own the support-telemetry exit while another owns metering. The lease is the correctness invariant: a database partial-unique index admits exactly one active lease per scope, a winner renews on a cadence (default lease around two minutes, capped well under an hour), and on the winner's outage the lease expires and the next acquire elects a successor. When there is only one cell, it simply sends direct, the whole mechanism degrades cleanly to a single box.
The send path and the holder's intake
A sender consults the egress route table for its (org, traffic class):
- Direct: this cell is the owner (or is single-cell); it opens the socket itself.
- Proxy: this cell is a peer; it forwards the bundle to the holder's intake endpoint, and the intake accepts a forward only from the current lease holder (a non-holder refuses). If the lease is stale or the peer is unreachable, the sender falls back to sending direct rather than lose the bundle, corp-side ingest is idempotent, so a rare double-send is safe while a dropped bundle is not.
How the owner is chosen: the election cascade
Choosing the owner is more than winning a race; it is choosing the residency- correct cell and only then racing for the lease. The election re-resolves the candidate set against live membership on every call (nothing is cached), so the answer tracks the fabric as it scales in and out:
Figure: the egress-owner election cascade. The selector is re-resolved against live membership every call; an unsatisfiable hard requirement or a set where no candidate can reach the destination is refused (red) rather than relaxed to "anywhere." The reachable candidates are ranked residency-correct (in-region preferred), and only the top-ranked local candidate attempts the lease race; every other cell defers (amber). The database lease is the single correctness invariant, topology is the preference in front of it, the race is the tie-break behind it. Ceiling: locality is a preference (PREFER), not a hard filter, unless the caller passes a hard requirement, a black hole is judged worse than a cross-region fallback, so an out-of-region cell wins only when no in-region survivor remains.
Table equivalent
| Stage | Condition | Outcome |
|---|---|---|
| Live re-resolve | Hard requirement matches no live cell | Refused |
| Live re-resolve | Empty candidate set | Refused |
| No-black-hole filter | No candidate can reach the destination | Refused |
| Ranking | This cell not in the set, or outranked by an in-region survivor | Deferred |
| Lease race | A peer acquired the lease first | Deferred |
| Lease race | This cell won the lease | Elected: owns the socket |
Failover follows from the same rule: because locality is a preference, the death of an in-region owner promotes the next in-region survivor on the next re-resolve, instead of the exit re-homing to whichever cell happens to race first.
Two-runtime parity
- The front-door resolver is at parity, and proven so. The inbound URL→node resolver is authored decision-for-decision on the core and the edge and pinned by a cross-language fixture generated from the core, so a divergence fails a test. See directory router.
- Egress-owner election is a core capability; the edge owes it. The residency-correct election across nodes is implemented on the core only. The Edge Agent is one device, not a cell with peers, and it deliberately disclaims cross-cell dispatch; it models the topology vocabulary and guards a per-device residency origin for its own outbound calls, but choosing an egress owner from a resolved set of peers is recorded as owed work, to be pinned by a cross-language fixture when the edge grows the seam. This is the parity ledger's discipline: land on one runtime, record the other as owed rather than narrow the claim silently. See core and edge.
Across the deployment matrix
| Corner | Behavior |
|---|---|
| Single-cell | The one cell elects itself and sends direct (unless a hard requirement it cannot meet forces a refusal). No lease contention. |
| Multi-cell | The election re-resolves live membership every call and races for the per-(org, class) lease; peers forward to the holder. |
| Sovereign / air-gapped | Ownership and reachability resolve against attested, leased peering: cell-local reads, no external directory. A destination with no reachable exit is refused, not black-holed. |
| Cloud-enabled | The cloud provider is a first-class selector dimension, so an owner can be chosen for provider-correct egress alongside region and zone. |
Related topics
- Directory router, the deny-closed resolvers behind the front doors, including the front-door URL cascade and its parity fixture.
- Cell network fabric, how cells find, trust, and reach each other beneath these front doors and leases.
- Core and edge, the parity discipline and the owed-work ledger the edge egress election sits on.
- Edge & fleet management, governed egress from the edge, and the fleet the elected exit serves.
- Proactive Support, one traffic class that rides an elected node egress.