Scale-out & distributed execution
A cell is a complete instance of the platform, and an estate is many of them. Scale-out is how one logical dataset and one logical query span those cells, sharded for capacity, placed with awareness of failure domains, committed atomically across cells when correctness requires it, and gathered back into one answer. It is the machinery beneath "the estate answers as one," and it stays inside the same governed substrate as a single-cell read.
In one paragraph
We shard with hash, range, and composite strategies; place shards with failure-domain awareness; run full two-phase commit across cells with coordinator and participant crash recovery; and scatter-gather a federated query across cells with signature-verified fragment results.
What scale-out provides
| Capability | What it does |
|---|---|
| Sharding | Hash, range, and composite strategies, with shard lifecycle, split, merge, and recovery. |
| Topology-aware placement | Primary and replica selection with failure-domain awareness, and rebalance planning driven by observability feedback (hot-shard detection, load distribution). |
| Two-phase commit across cells | Full 2PC with a coordinator and participants, and crash recovery on both sides: a distributed write commits atomically or not at all. |
| Workload-classified routing | A query is routed between the transactional and analytical engines by its workload class. |
| Replica shipping | Write-ahead-log shipping between shard replicas, with snapshot shipping, promotion, and split-brain fencing. |
| Scatter-gather federation | A query fans out across cells with a query codec and per-engine coordinators, and results are signature-verified as they return. |
| Resilience & governance | Failure detection, replica promotion, saga orchestration, consistency levels, write batching, buffer pool, result cache, resource governance, quota enforcement, a cost model, and predictive scaling. |
Underneath these sit the storage-layer building blocks that make a distributed read efficient: columnar file writers (Parquet, ORC, Avro) recording row-group and column statistics; Z-order and Hilbert clustering with an auto-optimizer; and durable persistence with a write-ahead log, snapshots, object-store offload, flash tiering, and a local cache.
Indexing is engine-native. An auto-index manager emits CREATE INDEX and
DROP INDEX through the shard-aware engine, so the underlying engines build and
maintain real indexes themselves rather than the platform keeping index structures
of its own beside them. That holds on the single-engine path and the shard-resolver
path alike. A cross-shard lookup on a column that is not the partition key is
served by fanning out and gathering over each shard's native indexes.
One plane is named and absent: index-based cross-shard pruning, deciding from an index which shards a query needs to touch at all, rather than asking all of them and combining. Fan-out returns correct answers without it, at the cost of touching shards that hold nothing relevant.
Tiering a transactional shard into the columnar tier
The transactional shard stays the write-of-record, and a flush pass tiers its committed rows into the columnar read tier: rows above a per-table high-water mark are written as a Parquet file in the owning workspace's object store, and an Iceberg snapshot is committed over that file.
The high-water mark is the part worth understanding, because it is what makes the pass safe to interrupt. It is not a separate store that can disagree with the data. It is a writer stamp carried inside the committed snapshot, so it advances in the same commit as the rows. A crash between a commit and the next pass therefore neither double-writes nor drops a row, and because the next pass reads the mark back from the table in object storage rather than from any cell, the pass resumes correctly after a cell fails over. Each pass costs the rows written since the last one, not the size of the table.
A cell flushes only the shards it is primary for. The pass needs no coordinator and no cloud, so it holds in the sovereign and air-gapped case, and a flush writes into the namespace of the database it came from, which belongs to exactly one workspace or one clean room, so it cannot cross that boundary.
A distributed query, end to end
A cross-cell query is split into fragments, dispatched to the cells that own the data, executed there against the governed engines, and gathered back with each fragment's result cryptographically verified, the data does not move to the query; the query moves to the data.
*Figure: the query moves to the data. A query is workload-classified and split into per-cell fragments; per-engine coordinators scatter them to the cells that own the shards, where each runs under local governance; on gather, every fragment's signature is verified before it joins the answer, and an unverifiable fragment is rejected. A distributed write commits through two-phase commit with crash recovery on both the coordinator and the participants.
Table equivalent
| Stage | Mechanism | Guarantee |
|---|---|---|
| Route | Workload-classified routing | Transactional vs analytical engine chosen per query |
| Split | Distributed planner + query codec | Fragments target the cells that own the data |
| Scatter | Per-engine coordinators | Execution happens where the data lives |
| Execute | Local governed engines | Same enforcement as a single-cell read |
| Gather | Signature-verified fragments | An unverifiable fragment is rejected, not trusted |
| Write | Two-phase commit + crash recovery | Atomic across cells, or not at all |
Why it is different
- Governance does not thin out at scale. A fragment runs on a cell's own governed engines, so a distributed read carries the same row filters, masks, classification, and lineage as a local one. See in-engine enforcement.
- Fragments are verified, not trusted. Scatter-gather checks a signature on each returned fragment, so a compromised or confused cell cannot inject a result into the answer.
- Cross-cell writes are atomic. Two-phase commit with recovery on both sides means a distributed write does not leave the estate half-updated after a crash.
Across the deployment matrix
- Single-cell, sharding and the storage-layer building blocks run within one cell; 2PC and scatter-gather are a no-op fast path.
- Multi-cell (elastic in and out). This is the plane's home: shards split, merge, and rebalance as cells join and leave, with topology-aware placement and split-brain fencing.
- Sovereign / air-gapped: every coordinator and participant is a cell in the estate; no external control plane is required to plan or commit a distributed query.
- Cloud-enabled: object-store offload and flash tiering are available as storage options beneath the shards.
Related
- Replication, DR & federation, the replication scopes and disaster recovery this rests on.
- The cell model, the unit scale-out composes.
- Scale-out storage & the lakehouse, the columnar formats and clustering beneath the shards.
- In-engine enforcement, the governance a distributed fragment carries.