Execution cores and access methods
FluxDB presents one uniform data-access surface and resolves it onto the right execution core underneath. This page is what those cores are, how they are tuned, and what sits on top of them.
Two execution cores, by workload class
| Core | Used for | Shape on disk |
|---|---|---|
| SQLite | Transactional, low-latency row operations | One .db file per shard, in WAL mode |
| DuckDB | Analytical, columnar scans and aggregations | One .duckdb file per shard |
The split is by workload class, not by product. A query is routed to the transactional or the analytical engine based on what it is, so an application does not choose an engine and then live with that choice.
How SQLite is tuned
The defaults are not SQLite's defaults. They are tuned for a multi-process cell, and each choice is deliberate:
| Setting | Why |
|---|---|
| WAL journal mode | Readers do not block the writer and the writer does not block readers, which is the concurrency posture a multi-process cell needs |
NORMAL synchronous | The durability-throughput point that pairs with WAL for this workload |
| A derived cache size | Sized from the environment rather than left at a fixed default |
mmap | SQLite reads straight from the page cache, without a syscall and a copy into user space. This is a direct latency reduction on the read path |
| A journal size limit | A cap on WAL growth. Without it a write burst grows the WAL and never returns the space, which is a real defect this setting exists to close |
Each shard gets its own connection pool with read and write lane separation, matched to the storage engine's concurrency model, plus resource governance, so one workload cannot starve another for connections.
How DuckDB is used
DuckDB is the analytical core, and it carries one property worth calling out specifically.
The same connection reads native storage or remote lakehouse tables. The analytical engine holds lakehouse bridge methods, so a single connection per shard can be pointed at native DuckDB storage or at remote Parquet with Delta or Iceberg metadata, through the corresponding scan extensions.
There is no separate reader engine. That matters because a second engine for "external tables" is how a platform ends up with two query planners, two security paths and two sets of behaviour for the same SQL. Here an external table is a table.
A catalog bridge converts Delta and Iceberg metadata into entries the analytical engine can attach directly, which is what makes reading tables in place a normal operation rather than an import.
The data access layer
Above the cores sits one resilient data access layer, and every service in the platform persists through it. That uniformity is what makes lifecycle, audit and replication behave consistently across the estate rather than per-subsystem.
It provides:
- Fetch, execute, batch execute and streaming iteration with chunking, so a large result does not have to be materialised whole.
- A fluent query builder.
- Transactions with automatic rollback, resting on a pinned-connection invariant so a transaction is genuinely one connection's worth of work.
- Connection pooling with read and write lanes, idle eviction, checkout timeouts, validation on checkout, and pool metrics.
- A connection budget, so pools are bounded rather than growing under pressure.
- Per-database write locking and project-root-relative path canonicalisation, so paths are deterministic and an in-memory database never accidentally becomes a file.
- Workspace database resolution and ownership, which resolves which physical database a logical workspace database maps to. This is what makes workspace scoping enforced beneath the query rather than above it.
- A statement interceptor and a function registry, the hooks governance and user-defined functions bind into.
- Planner statistics, feeding the cost-based decisions above.
The multi-model access methods
The same substrate presents several data models, each with a real backend rather than a convention over tables, and each reached through its own access method.
| Model | What it offers | Reached through |
|---|---|---|
| Relational | Tables, views and materialized views | SQL over the PostgreSQL wire, Arrow Flight SQL, GraphQL |
| Analytical | Columnar scans and aggregations over the same assets | SQL, routed by workload class |
| Document | Collections with filters, sort, skip, limit, distinct and bulk operations | The document API |
| Property graph | Labelled vertices and typed edges with properties, neighbours and traversal | GQL / openCypher, Gremlin |
| Vector | Upsert, batch upsert and similarity search | The vector API, and vector SQL functions |
| Time series | Path registration, single and batch ingestion, range and latest queries, aggregation, and pre-computed rollups | The time-series API, PromQL |
Alongside them, Structures is a distinct engine rather than an access method over the same cores: 34 structure types over a native Redis-compatible wire, part of the same data service.
Why one substrate matters more than the model count
Any platform can list data models. The consequence here is different, and it is the reason to run them together:
A document, a graph edge, a vector and a time series all carry the same workspace scope, the same classification, the same lineage record and the same cost attribution as a row in a table, because they are the same substrate under one access layer. There is no second governance path for the non-relational half of your data, and no second product to license, secure and reconcile.
The alternative, which is what most estates actually have, is a relational database plus a document store plus a graph database plus a vector database, each with its own authentication, its own access model and its own audit story, and four seams between them.
Related
- Scale-out and elasticity, how these cores spread across shards and cells.
- Storage and performance, where the files actually live.
- Structures, the distinct structure engine.
- Engines and access methods, the platform-wide view including object, lakehouse and messaging.