Skip to main content

Releases & versioning

An estate is many independent instances that were not started on the same day and will not be upgraded in the same instant. So the first thing to know about a System9 release is a design invariant, not a caveat: version skew across cells is normal, and every capability is built to tolerate it. A release is not a flag-day; it is a change that must remain safe while the estate around it is still on the old version.

Skew is normal: the two axes​

A cell is a complete instance with its own local databases, so skew appears on two axes at once, and both are expected:

  • New code against an older database: a cell running a newer build reads and writes a schema an older build created. The new code must not assume its own schema is already present.
  • One cell against a peer several releases ahead (or behind): cells in one estate coordinate as peers without being pinned to a single version. Ephemeral cells make this routine: roughly a hundred are launched at a time for a job, each its own instance, often against permanent peers on a different release.

This is the same version-skew discipline the rest of Platform Architecture returns to. A change that assumes every cell is on the same version is a change that breaks a real estate.

What keeps a change safe​

Figure: the expand / migrate / contract discipline. A schema or wire change is made in three ordered phases so that old and new coexist: expand (cyan) adds the new shape while the old still reads; migrate backfills and dual-writes while both versions run; contract (green) removes the old shape only after every reader has moved. Independently, a wire-schema pin (amber, dashed) refuses a peer speaking a different wire version with a typed error, so skew never silently corrupts data.

Table equivalent

PhaseWhat happensWhy it is safe under skew
ExpandAdd the new schema or field; leave the old in placeAn older cell still reads; a newer cell can write both
MigrateBackfill and dual-write; both versions coexistNo reader is stranded while the estate catches up
ContractRemove the old schema or fieldRuns only after every reader has moved to the new shape
Wire-schema pinEach call stamps its wire-schema versionA peer on a different version is refused with a typed error, not silently mis-parsed

Two mechanisms make this concrete:

  • Wire and schema changes are expand / migrate / contract. A change lands the new shape first, moves data while both shapes are readable, and only removes the old shape once nothing reads it. This is what lets new code run against an older database without a coordinated flag-day.
  • Schema-pinned, mutually-authenticated calls. Every inter-cell request stamps the wire-schema version, so a peer speaking a different version is rejected with a typed error rather than corrupting data. On the Edge Agent, s9 schema-hash prints the vendored wire-schema hash, which is exactly what you compare when diagnosing a version-skew rejection between a device and a peer.

Safe upgrade: how a release rolls out​

An upgrade is delivered as a signed release, not an ambient package pull. Applying it is governed and reversible:

  • Signed manifests. A release is a signed manifest: files with their hashes and modes, service definitions, and the target architecture, OS, and variant, verified against the trust roots a cell or device already holds. An unsigned or altered release does not apply. Per-release notes ride with that signed manifest rather than being fetched from an unauthenticated changelog.
  • Ring-by-ring, with a freeze. Releases move through tracks and named cohorts in a ring-by-ring rollout, so a change proves itself on a small ring before it reaches the fleet. A change freeze holds a cohort where it is during a sensitive window.
  • Atomic swap, precheck, post-test, rollback. An apply runs a precheck, swaps atomically, runs a post-test, and rolls back on failure, the device does not come up half-upgraded. Drift detection reconciles a device that has fallen out of its assigned state.
  • Store data passes. After code moves, a cell or device runs its outstanding store migrations. On the edge, s9 migrate --dry-run reports what would change before s9 migrate applies it, safe to run on a production device.

Across the deployment matrix​

  • Single-cell: an upgrade is a local, signed apply followed by the store data passes; there is no peer to skew against, but the same expand/migrate/contract discipline still protects new code reading an older local database.
  • Multi-cell: cells upgrade independently and coordinate across the skew; the wire-schema pin keeps a mixed-version estate correct rather than requiring a synchronized cutover.
  • Sovereign / air-gapped: a release is carried as a signed offline bundle and verified against held trust roots with no network (s9 fleet-releases apply-bundle). Upgrades reach a dark site by courier, not by phone-home.
  • Cloud-enabled: identical machinery; the release ring simply has a network to reach the next cohort over.
Design status

Versioning and the skew invariant are described by mechanism. The expand / migrate / contract discipline, the wire-schema pin with typed rejection, and the signed, ring-by-ring, rollback-able rollout are the shipped model. This page is the versioning model; the specific contents of any single release travel with that release's signed manifest.