Skip to main content

Time series

Sensor readings, metrics, telemetry and market ticks are not shaped like rows in a table, and forcing them into one is how a data platform ends up with a separate time-series product beside it carrying its own security model.

System9 carries a tiered time-series engine as one of the engine families on the same governed substrate, so a series is catalogued, classified, workspace-scoped and policy-enforced exactly like a table.

The tiers​

A series is addressed by a path, registered before ingest, and samples move through three tiers as they age.

TierWhat it holdsForm
HotRaw samples, keyed by pathIn memory, fronted by a write-ahead log
WarmRollups aggregated from hot-tier samplesFrames
ColdFlushed framesParquet bytes, an open format

The aging path is the interesting part. Rollup is not a query-time trick; it is a tier transition. Hot samples are aggregated into warm frames, and warm frames flush to Parquet for cold persistence, so a long-range query reads pre-computed aggregates rather than re-scanning raw samples it did not need.

A write-ahead log sits in front of the hot tier, so an in-memory tier does not mean an unrecoverable one.

The hot tier is in memory

The engine's storage is in-memory with no external dependency required, which is what lets it run in a cell with nothing else installed, including at the edge. Durability comes from the write-ahead log and from cold-tier Parquet, so size the hot tier for the window you actually query hot.

What you can do with a series​

OperationNotes
Path registrationA series is declared before it is written to
Single and batch ingestionBoth paths supported
Range queriesOver a time window
Latest queriesThe current value without scanning
AggregationOver a window
Pre-computed rollupsServed from the warm tier rather than recomputed

A tap exposes the stream of arriving samples for downstream consumers, and a materialisation target lets a pipeline land its output as a series rather than as rows.

It is governed like everything else​

This is the reason it belongs in the data platform rather than beside it.

The external time-series face is not a shortcut around policy. A request authenticates with a per-database credential that resolves to a directory entity, passes a rate limiter scoped to that caller, resolves a grant, and then runs the same data-security evaluation as any other engine: rows are filtered and columns are masked from the policy result before anything is returned. A denial is audited rather than silently returning an empty set.

So a series carries:

  • a workspace scope, enforced beneath the query;
  • a classification from the same five-rung ladder;
  • row filters and column masks compiled from the same policy;
  • a lineage record and a cost, like any other read.

Where it shows up​

  • Metrics and telemetry, as the shape the observability plane records into.
  • Sensor intake, where readings arrive with clock synchronisation across sensors and a cognition policy governing what may be inferred from them.
  • Predictive maintenance, where condition indicators are computed over series, on the core or on the device.
  • Structures, which also carries a time-series structure type for the cases where the natural home is the structure engine rather than a registered path.