Skip to main content

Key Custody and Lifecycle

Sovereignty is only real if the customer's keys never have to leave the customer's boundary. System9 is built so that private key material can be generated locally, held in hardware, and used without being exported, and so that a disconnected cell can still issue and validate the certificates it needs.

Where private keys live​

Key material can be held in software or behind hardware, selected per key:

BackendNotes
Software (file)Private key persisted with restrictive filesystem permissions (mode 0600 on Unix).
TPM 2.0TPM-sealed vaults; signing without exporting the key.
PKCS#11 / HSMStandard HSM interface; the private half never leaves the module boundary.
YubiKey PIVHardware token custody for operator and device keys.

On a device, the private key is generated on-device (ECDSA P-384) and never transits the wire, enrollment sends only a certificate signing request. A hardware fingerprint binds the identity to the physical device, so a second enrollment from the same machine is refused unless the earlier record is decommissioned. See Identity & Access for the full enrollment flow.

Sealing and the key hierarchy​

Beneath the individual certificates sits a key-encryption-key (KEK) hierarchy with epochs and a persistent epoch record, a hash-chained ratchet log, and a background re-wrap worker for migration. A forward-secret KEK ratchet is built on top of these, it rolls a flavor's KEK, appends to the tamper-evident ratchet log, and re-wraps every key sealed under the retired KEK, but it is opt-in and off by default, so forward secrecy is not asserted as a live guarantee here. Cell identity keys, node identity keys, and node sealing keys are separated by purpose, per-purpose key separation means a key used for one function cannot be repurposed for another, and material that warrants it is held under high-sensitivity custody. A secrets anchor, with an anchor eraser, roots the local secret store.

Secrets and external keys can also resolve through the customer's own systems: pluggable secret backends (local, AWS Secrets Manager, AWS Parameter Store, Azure Key Vault, GCP Secret Manager) and key-management bridges to AWS KMS, Azure Key Vault, GCP KMS, and HashiCorp Vault, with a bring-your-own-key registry.

Rotation, revocation, and consistency​

Certificate lifecycle is automated and overlapping rather than disruptive:

  • Refresh. A device generates a fresh keypair and CSR and requests a new leaf certificate whose validity begins partway through the current certificate's life, so there is an overlap window rather than an expiry cliff. Any subset of the X.509, SSH, and PGP material can be refreshed independently.
  • Delegation rotation. Permissions, scope, or lifetime can be changed without re-enrolling, inside a short grace window, so authority changes do not require a certificate rotation or a reboot.
  • Revocation. Revocation spans X.509, SSH, and PGP, with a gossip revocation bus for propagation and pre-flight checks on the verifying side. On the device, a revoked delegation halts the daemon; because delegation is bound to the certificate fingerprint, revoking the delegation invalidates every future request from that device without a certificate rotation.
  • Consistency. Rotation is coordinated as an atomic unit across peers, with a material registry and gossip-based consistency checking that verifies every peer converged on the same material.

Revocation material is delivered through CRL, OCSP, and SSH KRL. A dialing cell pre-flight checks these before trusting a peer, and a serving cell staples its own cached OCSP response onto its TLS listeners so a connecting peer gets the freshness proof inside the handshake, served today, and fail-safe to unstapled when a leaf has no current staple (see revocation on the wire). In the sovereign, air-gapped corner delivery rides the same transport the rest of the device's state travels over. A signed revocation bundle is carried on removable media and applied with no network reachable, verified against the device's pinned trust roots before anything in it is believed. A connected device additionally runs a background refresh that walks each registered issuer on a short tick, so a revocation published on a cell reaches the device's cache quickly; a fetch that fails is logged and the previous cache entry stands rather than the daemon failing.

Offline and tactical custody​

A disconnected cell is provisioned with pre-provisioned certificates and a subordinate CA, so it can issue and validate with no upstream reachable. Licence enforcement uses signed offline files verified with no phone-home on the enforcement path, and the licence private key never leaves an HSM.

For tactical profiles, policy-driven zeroize can render local key material useless on detectable conditions (for example, a heartbeat-loss threshold on a profile that declares one). Most device profiles declare no such trigger and never auto-destroy.

Software-only zeroize is not capture-resistant

Zeroize on a software-only path cannot fire on a device that has already been powered down and carried away. Capture-resistant sanitization requires hardware anti-tamper, which is an integration point rather than a capability of the software path. What the software path provides is policy-driven zeroize on detectable conditions.

Threshold custody and escrow​

For material where a single custodian is too much trust, System9 supports Shamir secret sharing with an m-of-n signer and key ceremonies that collect approvals before a secret is reassembled, plus escrow with legal-hold records so recovery needs a quorum rather than one person.

How a revoked status survives a restart, and the one ordering caveat

The durable, authoritative revoked state lives in the per-algorithm revocation tables a revoke call writes, and those tables replicate convergently. The in-process lookup every crypto-using service consults before trusting freshly validated material is a rebuildable projection of them, indexed by issued credential id, by X.509 serial for the OCSP responder and CRL builder, and by public-key hash for the SSH revocation-list builder and the raw-key validator.

That projection is rebuilt from the durable tables at service start and again on a reconcile tick, so a restarted cell, a cell that has just joined, and a cell that healed from a partition all answer from their own rows rather than waiting for a peer to re-gossip. A cache miss means ask the responder, never good, which is what keeps a credential that has not been gossiped yet from being treated as valid.

One caveat applies to the air-gapped courier lane: a revocation bundle carries a signed sequence number, and the device does not enforce ordering on it. Enforcing it would need a counter shared across every cell minting into one scope, and cells mint clock-derived sequences with no shared order, so an ordering rule would refuse a legitimate bundle the moment a second cell minted one, fail-closed, which on this path means a burned key staying valid on a dark device. What defends against a replayed, reordered or truncated bundle instead is that the merge is monotone: a serial this device has been told is revoked stays revoked, and a later bundle that omits an entry does not un-revoke it. A bundle can only ever add revocations.

Design status

Key material is described by mechanism, not by grade; no component is presented here as certified. Post-quantum hybrid X.509 issuance is built and opt-in: the issuance path dispatches on the crypto policy's algorithm_family, hybrid and pq_only mint a classical + ML-DSA-65 certificate and fail closed (no downgrade), and classical is the default. See Post-quantum hybrid signatures. FIPS 140-3 and CNSA 2.0 policy profiles with fail-closed startup and runtime gates exist as algorithm policy; a compliance report's readiness statement is not a certification claim.

Deployment caveat: set your PEN before hybrid certs leave the fabric

The default post-quantum OID arc is a reserved placeholder (PEN 99999, unregistered) and is runtime-warned. A deployment must set its own registered Private Enterprise Number arc before any hybrid certificate leaves the fabric. The classical half stays valid regardless.