Skip to main content

Data Platform

A full data stack, carried by every cell, on one governed substrate: a catalog, several engine families, open lakehouse formats, object storage, a durable message bus, streaming, and pipelines.

Because it is one substrate rather than a set of integrated products, a read already carries its lineage, its classification, its row filters and column masks, and its cost. Not bolted on afterward, and not skippable by reaching an engine a different way.

Start here​

PageWhat you get
ConceptsThe four ideas the rest of this topic is a consequence of
Resources you provisionThe inventory of what you actually create, and the five properties every one carries
Engines and access methodsThe engine families and the protocols your existing clients already speak

One product, several engines​

The Data Platform is one product and one service. You provision inside it, govern it, meter it and audit it as a single thing, through one catalog and one enforcement plane.

Underneath, it runs several purpose-built engines, and knowing which is which matters when you are choosing where data should live, not when you are buying.

Figure: one catalog over several engines, resting on two storage planes with different jobs. The hot path carries live shards and memory-speed structures, where latency is the whole point. Object storage carries the lakehouse, aged time-series tiers and shard backups, where durability and cost are. The dotted edges are the deliberate crossings: FluxDB reaches the object plane for lakehouse tables and backups, not for its live SQL path. The engines differ in what they are good at; the governance over them does not differ at all.

Table equivalent

ComponentWhat it isWhere its bulk data rests
FluxDBThe core component: a SQL multi-model database over SQLite and DuckDB, with document, graph, vector and time-series access methods, sharded and placed across cellsLow-latency local shard files. Object storage only for lakehouse tables and shard backups
StructuresA memory-speed command engine with 34 structure types and a Redis-compatible wire, durable through its own write-ahead logHot in memory, spilling warm and cold
Time-series engineA tiered engine fed from the stream path, with rollups from hot samples into warm framesParquet, cold tier
LakehouseIceberg and Delta as co-equal open formats, read zero-copyObject storage, in open format
Object storageThe durable plane every engine rests on: content-addressed, deduplicating, erasure-codedItself
MessagingThe durable bus. Persists through FluxDB and tiers to object storage, sharing the connector layer with ingestionFluxDB, then object storage

Two things hold across all of them, and they are the reason this is one product:

  • One catalog. Every asset, whatever engine holds it, is catalogued, classified, lineage-tracked and searchable in one namespace.
  • One enforcement plane. Row filters and column masks compile into whichever engine executes, so the policy does not change when the storage does.

The stores in depth​

AreaWhat you'll find
FluxDBThe database system: execution cores, access methods, scale-out and elasticity, and the two storage planes
Catalog & lakehouseThe control plane: one namespace over every engine, an ontology over it, governed promotion, discovery, plus Iceberg and Delta at format parity and the virtual graph
StructuresThe memory-speed engine: the type surface, the Redis wire, concurrency and expiry, persistence and cross-cell distribution
Object storageContent-addressed, deduplicating, erasure-coded objects where classification is enforced in the data path and durability is a checkable claim
MessagingA durable bus where a message carries the same classification, scope, and audit chain as a table
Time seriesA tiered engine with a hot tier, rollup into warm frames, and cold persistence as Parquet
Streaming & real-timeComplex event processing, streaming SQL, KQL and PromQL, live queries, a reflex engine, and hardening for classified work

The work​

AreaWhat you'll find
Data governanceOne enforcement plane for row filters and column masks, classification and lineage, schema governance, data lifecycle, and the authoritative deny
Data engineeringPipelines across cells, notebooks, and functions including AI functions invoked from SQL
Modeling, products & BITransformation projects, the semantic layer, data products with typed ports and contracts, and every surface a reporting tool connects over
Data ExchangeA sovereign cross-organization marketplace where a listing is a signed attestation verifiable offline
Scale-out & distributionSharding, topology-aware placement, two-phase commit across cells, and scatter-gather federation

Why it is different​

  • Governance is part of the query, not a wrapper. Row filters and column masks compile into the plan of seven engine types. The result carries a classification, the read is lineage-captured, and the cost is attributed. There is no second door.
  • Your clients connect unchanged. PostgreSQL wire, RESP, Gremlin, GQL, GraphQL, Arrow Flight SQL, Iceberg REST, Delta Sharing, S3, WebHDFS, SFTP. Every one of them re-enters the same authorizer.
  • The same governance covers a message, an object, and a table, because they are the same substrate. This is the one that is hard to retrofit.
  • Structured and unstructured converge. A scanned PDF is decoded, classified, embedded, and registered as a catalog asset, so it inherits the same classification ladder and policy surface as a table.
  • One authoritative deny. A revoke on the canonical governance edge stops serving at every face: the marketplace, Delta Sharing, and the Iceberg REST catalog alike.

The question this lets you ask​

When your compliance team asks which systems hold personal data, does the answer include the file shares, the mailboxes, the scanned PDFs, and the video, and can you enforce a policy on them the same way you enforce it on a table?