Audit and Tamper-Evidence
System9's audit records are hash-chained: each record links to the one before it, so inserting, reordering, or mutating a record breaks the chain and is detectable. The design goal is not a bigger log. It is a record whose integrity can be checked mathematically rather than trusted.
The platform does not have "an audit log." It has roughly sixteen independent tamper-evident chains, each answering a specific question an auditor asks, and each recording its own gaps as first-class events rather than presenting an unbroken surface that hides them.
Figure E2: a tamper-evident chain, and the field of sixteen. Each record (green) carries the hash of the one before it, so mutation in the middle propagates forward and is detectable. A counted gap (amber, dashed) is a recorded append-failure event, so a verifier can tell a known append failure from a removed row. There are roughly sixteen such independent chains.
Table equivalent, the sixteen chains and what each proves
| Chain | The question it answers |
|---|---|
| Workspace activity ledger | What happened in this workspace? |
| Metering audit chain | Is this invoice line real?: with gap detection |
| Lineage chain (per cell) | Where did this data come from? |
| Workflow journal | What did this agent do, and can I re-run it? |
| Decision audit chain | Why was this action allowed or refused?: co-witnessed by peer cells |
| Crypto ledger Merkle audit | Was the key material tampered with?: external witness bundles |
| Certificate transparency log | Was a certificate issued behind my back? |
| Policy commitment | Has this policy set changed since it was signed? |
| Storage audit chain | Was an object access removed from the record?: with counted gaps |
| Stream chain of custody | Who touched this record, in what order? |
| Clean-room activity ledger + signature chain | What ran in this room, approved by whom? |
| Edge device journal | What did this device do while it was offline? |
| Capture hash chain (edge) | Is this image or recording the original? |
| Job ledger | Who dispatched this work, and where did it go? |
| Configuration attestation chain | What is this cell actually running?: signed by the cell |
| Coding diff receipts (per run) | Which agent wrote this line, and has the diff changed?: verifiable offline |
How a chain is built
Chains are append-only and forward-linked. Each record carries the hash of the previous record and a hash of its own contents, so the chain has a verifiable tip and any tampering in the middle propagates forward. Several signing models are used, matched to what each chain must prove:
| Model | Where it is used | Property |
|---|---|---|
| SHA-256 inter-record link | Security events, the activity ledger, most newer chains | Detects insertion, reordering, and mutation. |
| Per-record HMAC + signed chain head | Metering usage records | The head is verifiable off-cell with the public key. |
| SHA-256 link + rolling HMAC checkpoint | Messaging | A checkpoint catches retroactive tampering even after key loss. |
| Ed25519 signatures | Licences, telemetry events, metering peer bundles, device records | Third-party-verifiable provenance. |
Every linked chain is held to the same contract: append-only with no update or delete, a linked tip, a forward iterator for re-anchoring, end-to-end re-hash verification, and a fail-closed write path.
Counted gaps
An append-only file's absence proves nothing, a missing row could be a lost write or a deliberate deletion, and you cannot tell which. System9 records a chain-append failure as a first-class gap event, so a verifier can distinguish a known append failure from a removed row. This gap-counting property is rare and it matters: it is what lets a chain be honest about its own gaps rather than presenting an unbroken surface that hides them. For metering, a detected chain gap feeds the enforcement ladder rather than being swept aside.
What an outside party can verify
- The activity ledger is append-only, hash-chained per workspace, and carries a verification endpoint with an HMAC checkpoint anchor. A clean-room-scoped sibling ledger has the same shape.
- The metering chain head is signed and can be verified off-cell by a party holding only the public key.
- The crypto ledger supports external witness proof bundles, an outside party can hold evidence that the ledger was in a given state at a given time, and signed policy commitments let a policy set be proven unchanged.
- Merkle inclusion proofs let a recipient confirm a signed artifact was logged: the Edge Agent ships a pure-Rust inclusion-proof verifier that mirrors the cell's reference implementation byte-for-byte and is covered by known-answer tests. This transparency log is internal. It is deliberately not published to a public log, to avoid leaking internal hostnames, while remaining offline-verifiable.
Symmetric by design
The architecture's goal is a symmetric audit, every participant observes and is observed, with decisions co-witnessed by peer cells and cells attesting their own configuration into the audit stream.
Data-subject scope
A data-subject erasure or access request is bounded to the named subject: a store that cannot scope to a single subject is skipped and named rather than wiped. Coverage is partial (only registered providers are reached) so scope is enforced while completeness remains in progress. Right-to-erasure is not complete.
Related topics
- Metering and attribution, the metering chain and its off-cell verification.
- Signing & Cryptography, the signatures and Merkle proofs underneath.
- Governance (Deny-Closed), the decisions these chains record.