FluxDB
FluxDB is a database system. Not a thin wrapper over somebody else's database, and not a metadata layer above one: a multi-model engine with its own access methods, its own scale-out plane, its own placement and routing, and its own governance compiled into execution.
It is deliberately not System9-specific. FluxDB is the database system the platform runs on, and it is referable on its own terms: when this documentation says a workspace's tables, documents, graphs, vectors and series live in FluxDB, that is a statement about a database, not a euphemism for "our storage layer."
The name is the architecture. Flux is multi-model and changing shape: rows, documents, graphs, vectors, time series and analytical columns over one substrate, moving between cells as the workload moves.
The building-block decision, stated up front
FluxDB uses SQLite as its transactional execution core and DuckDB as its analytical one. That is a deliberate architectural choice and it is the first thing to understand about the system, because it is the thing most likely to be misread.
Embedded does not mean small, and battle-tested does not mean slow.
SQLite is the most widely deployed database engine in existence, running in every phone, every browser and a large fraction of the world's applications, with a test suite proportionally larger than almost any codebase in the industry. DuckDB is a vectorized columnar analytical engine built by database researchers specifically to do OLAP work in-process, and it reads Parquet natively.
These are two of the fastest, most rigorously exercised engines ever written, and building on them buys three properties that are very hard to get any other way.
1. The query does not cross a network to reach its data
This is the one that matters most and is the easiest to underestimate. In a conventional architecture, a query travels from your process to a database server over a socket, is parsed there, reads from that machine's storage, and sends results back.
In an embedded engine, execution happens in the same process as the caller, against a local file. There is no network hop, no connection handshake on the hot path, no serialization to a wire format and back.
That is a latency floor a client-server architecture structurally cannot reach, and it is why the per-shard unit of FluxDB is fast before any of our own work is counted.
2. No external database licence, no vendor control plane
A cell carries its own engines. There is nothing to license separately, no external service to be reachable, and nothing to phone home. That is what makes sovereign and air-gapped operation the default mode rather than a deployment exception, and it is why a cell boots into a complete data platform on a disconnected network.
3. Correctness we did not have to earn
SQLite's durability and transactional correctness are among the most heavily exercised properties in all of software. DuckDB's vectorized execution is the subject of published database research.
Building an execution core from scratch would mean spending years re-earning guarantees that already exist, and being less trustworthy at the end of it. The interesting engineering is not the execution core. It is everything around it, and that is where the work went.
What FluxDB adds
The engines execute. FluxDB is the system that makes them a database platform.
| Layer | What it adds |
|---|---|
| Access methods | One uniform data access layer, and every way the substrate is reached: SQL, document, graph, vector and time series, so a workload picks its shape rather than its product |
| Scale-out | Sharding with several strategies, topology-aware placement, a query router with pushdown and parallel execution, two-phase commit across cells, and elastic split and merge |
| Storage | Live shards on a low-latency local path, kept deliberately off the object plane, with object storage reached through the lakehouse and for shard backups |
| Governance | Row filters and column masks compiled into the plan of every engine, a workspace scope resolved beneath the query, lineage capture, and cost attribution. See in-engine enforcement |
The access methods
A database is only as useful as the ways you can talk to it. FluxDB is reached through the protocols your existing clients already speak, and the governance is identical on every one of them, so picking a different access method never means picking a different policy.
| Access method | Protocol or surface | What connects over it |
|---|---|---|
| SQL, transactional and analytical | The PostgreSQL wire protocol | psql, ODBC and JDBC drivers, Power BI, Tableau, and essentially any SQL tool |
| Columnar bulk read | Arrow Flight SQL | High-throughput analytical pulls, with multi-cell fan-out |
| Graph | GQL / openCypher and Gremlin | Property-graph traversal, including the zero-copy virtual graph over lakehouse tables |
| Document | The document API | Collections with filters, sort, skip, limit, distinct and bulk operations |
| Vector | The vector API, and vector SQL functions | Upsert, batch upsert and similarity search, reachable from SQL as well |
| Time series | The time-series API and PromQL | Range and latest queries, aggregation, and pre-computed rollups |
| Merged domain schema | GraphQL, with query cost analysis | Application access across domains in one call |
| Lakehouse tables | Iceberg REST and Delta Sharing, and plain SQL | Engine-neutral access to the same tables, read in place |
The two that matter most in an evaluation are the first two, because they are the ones that need no adapter written. A SQL tool that can talk to PostgreSQL can talk to FluxDB, and a tool that speaks Arrow Flight SQL gets columnar throughput without a translation layer in the middle.
Performance
FluxDB is fast for a reason that is easy to state and hard to copy: it is built on two of the fastest database engines in existence, and then it multiplies them.
The engines are not the compromise. They are the advantage
There is a reflex that reads "embedded" as "small." It is worth spending a moment on why the opposite is true here.
SQLite is the most widely deployed database engine on the planet, running in every phone, every browser and a vast share of the world's applications. Its performance and durability characteristics are among the most heavily exercised and most publicly documented in all of software.
DuckDB is a vectorized columnar analytical engine built by database researchers specifically to do OLAP work in-process. It executes across cores in parallel, spills beyond memory when a workload demands it, and reads Parquet natively. On the public single-node analytical benchmarks that the industry actually runs, it sits at the front of the field.
Both are independently benchmarked, continuously, by people with no stake in this platform. You do not have to take our word for their speed, and you should not have to. That is precisely why they were chosen: the execution core is the one part of a database you should not be asked to trust on a vendor's say-so.
The in-process floor
In a conventional architecture a query leaves your process, crosses a socket, gets parsed on another machine, reads that machine's storage, and sends results back. Every one of those steps costs.
In an embedded engine, execution happens in the caller's own process against a local file. No network hop. No connection handshake on the hot path. No serialization to a wire format and back.
That is a latency floor a client-server database structurally cannot reach, no matter how well it is tuned or how much hardware sits behind it. It is not a tuning advantage. It is an architectural one.
Then we multiply
Here is the part that changes the category. A conventional deployment inherits its engine's single-node ceiling. FluxDB does not, because a shard is not a slice of one server's resources. It is a whole engine.
Add a shard and you add an entire execution core. Add a cell and you add its compute, its networking and its storage to the pool. Every cell in an estate contributes all three, anywhere in the world, across physical and jurisdictional boundaries, to one system.
| What you add | What you get |
|---|---|
| A shard | Another complete embedded engine, executing in-process against its own local storage |
| A cell | Its full compute, network and storage capacity, plus every engine running on it |
| A service group | A durability and placement boundary over those cells, spanning zones, regions and clouds |
So the ceiling is not a machine. The ceiling is how much hardware you decide to point at the problem, and the placement engine, router and rebalancer are what make adding it an operation rather than a migration project.
What keeps fan-out from eating the gain
Distribution only helps if the distributed parts do the work. The router pushes predicates and projections down into each shard, so a scatter-gather returns the smallest useful result rather than dragging raw rows back for a coordinator to filter. Reads route to replicas; writes route to the owning shard; a query that resolves to a single shard pays no coordination at all.
On top of that the system tunes itself: an adaptive optimizer adjusts plans against observed behaviour, an index advisor maintains indexing rather than leaving it to be guessed at during an incident, and cache warming means a cold shard is not cold on first use.
The question that actually decides a deployment
In practice, raw single-query latency is rarely what constrains a real estate. Most workloads run on considerably more hardware than they need, and the engines underneath this one are already fast enough that the query is not the problem.
The problem is what happens as the data gets larger, and as it spreads across sites, regions and boundaries you do not control. That is a question about scale, throughput, placement and durability, and it is the question this architecture is built to answer:
- Throughput scales with shard and cell count, not with one server's ceiling.
- Placement is topology-aware, so data sits where residency, latency and cost require.
- Durability is configurable per workload, from mirrored copies to Reed-Solomon erasure coding, without bolting on a second system.
- Reach extends to sites and devices a centralised warehouse cannot serve at all.
How to size it
Start from the engines, because you can measure them yourself and because public third-party numbers already exist for both.
- Characterise one shard on your hardware, for your workload. This is an ordinary SQLite or DuckDB measurement, and the published work on both is a reasonable starting estimate before you run anything.
- Multiply by the shard plan. Adding shards adds engines.
- Subtract only the coordination you actually need, which for single-shard queries is none, and for scatter-gather is only the rows that survive pushdown.
That model is worth more than any single headline figure, because it tells you the thing a headline figure never does: what changes when you add hardware.
What runs on FluxDB, and what does not
FluxDB is the persistence and execution layer most of the platform's data services are built on. One major engine is deliberately not among them, and conflating the two is the most common misreading, so it is worth being exact.
| Consumer | How it uses FluxDB |
|---|---|
| Catalogs, tables and assets | The obvious one: what you provision in a workspace |
| The multi-model stores | Documents, graphs, vectors and time series, through their own access methods |
| Messaging | The durable bus persists through FluxDB into its own logical database, and reuses the same tiering, convergence and connector primitives. Messaging is a data service, not a neighbour of one |
| Streams | Named workspace-scoped streams and the real-time query surfaces over them |
| Every platform service | Each persists through the same access layer, which is what makes lifecycle, audit and replication behave consistently across the estate rather than per-subsystem |
Structures does not run on FluxDB. It is a separate memory-speed, Redis-class command engine with its own store, command processor, filesystem write-ahead log and slot-based sharding, durable and distributed in its own right. The separation is enforced: a guard test fails the build if the structure engine imports FluxDB, with one documented exception for a shared SQL-evaluation seam.
They are both components of the data service, provisioned, governed, metered and audited identically. The reason to know they are different engines is practical rather than commercial: they have different durability and latency characteristics, so you size and choose between them on those grounds.
What is measured
The data service publishes the signals that tell you whether the data plane is keeping its own commitments, not just whether the process is up.
Convergence is instrumented directly: unmerged keys, peers tracked, and the age of the oldest unmerged sample. That last one is the honest measure of convergence health, because a count of outstanding merges says nothing about whether the backlog is seconds old or hours old.
Alongside it: catalog leaf provisioning, registered commands, command audit emit failures (a governance signal, not an operational one, since a command that ran without landing in the audit record is a gap worth alerting on), structure load factor, false-positive and depth ratios for probabilistic structures, memory statistics, and per-database file sizes.
Every query also lands in the universal database instruments that every service carries, query count and query duration, so load attribution across services is available without anyone instrumenting a call path by hand. See what is measured.
Where FluxDB sits
FluxDB is the user-facing data plane: the catalogs, tables, structures, streams and models you provision inside a workspace run on it.
It is distinct from the cell's own internal state. A cell partitions its own bookkeeping into nine logical databases that you never provision, size or query, and those are architectural plumbing rather than a surface you manage. FluxDB is the engine beneath both, but only one of them is yours.
In this section
| Page | What it covers |
|---|---|
| Execution cores and access methods | How SQLite and DuckDB are used and tuned, and every access method over them |
| Scale-out and elasticity | Sharding, placement, query distribution, distributed transactions, and scaling in and out |
| Storage and performance | The two storage planes, their security, and how to reason about capacity |
Related
- Engines and access methods, the protocols these are reached over.
- Scale-out and distribution, the distribution plane in the Data Platform view.
- In-engine enforcement, the policy compiled into every plan.
- Object storage, the other storage plane.