Bring up a node
A cell is a complete instance of the platform: its own identity, its own keys, its own local SQLite databases. A node is the identity that a group of cells belongs to. Bringing up a node therefore means standing up its first cell (which mints the node), and, for a fleet, enrolling further cells under that node.
System9 ships as a sealed, licensed machine image, so bring-up is booting that image and completing setup. This runbook covers the four deployment scenarios, and each is the same entrypoint and setup flow under a different configuration and connectivity posture, because a capability the product is sold on has to hold in every corner rather than one.
Read the quickstart first if you have not stood one up before. This page assumes it and goes further: multi-cell, air-gapped, and what to do when a step fails.
s9s9-admindrives a cell / node (the Python core),python -m app.cli.main. Used in this guide fortopology,replication, andworkspace.s9drives an Edge Agent (the Rust device binary):enroll,run,chat.
They are different programs on different hosts. This guide's node commands are
s9-admin; the edge commands are s9.
The bring-up path, once
Every scenario walks the same three stages. What differs is where the config points and whether a link exists.
Figure: the bring-up path. The orchestrator starts the services; a setup gate
holds authenticated routes at 503 (amber) until the setup flow (cyan) mints the
org, node, cell, administrator, crypto, and activates the fabric; then the cell
serves (green). Only a multi-cell node adds the enroll stage (filled). Ceiling:
the flow is license-driven; a development image can boot unlicensed, a production
image requires a signed license or a signed join bundle.
Table equivalent
| Stage | What happens | Where |
|---|---|---|
| Orchestrator start | backend/main.py start launches services dependency-ordered | The host |
| Setup gate | Authenticated routes return 503 until setup completes | node front door |
| Setup flow | License → org → node → cell → admin → crypto → fabric activate | /api/public/setup |
| Serving | The cell serves work on the node front door | Ports 8080 / 8443 |
| Enroll (multi-cell) | Further cells join via a signed join bundle | Cell-join ceremony |
Before you start: the bootstrap token
This is the step most first attempts miss, so it comes first.
Setup is authorised by a bootstrap token, and nothing hands it to you over the network. There is deliberately no endpoint that issues one: an earlier version gated token issuance on the caller's address, which behind a proxy or ingress means every pod in the cluster, and behind a load balancer means the internet. Provisioning authority is not network reachability.
Read it out of band, from whichever of these your deployment uses:
| Source | Where |
|---|---|
| The startup banner | Printed by the cell on first boot |
| A file on the host | config/bootstrap_token, a 64-character hex string |
| Cloud-init | Injected at instance creation |
| A signed provisioning bundle | Delivered with the image |
Then supply it to setup. A request without it is refused, and that refusal is the expected behaviour rather than a fault.
Scenario 1: A single cell
The normal path. One image, one licence, one cell that mints its own node.
What you need
| A host | A cloud instance or VM that can run the cell image |
| A licence | S9_LICENSE_KEY, or a file at config/license.s9lic |
| An administrator password | S9_ADMIN_PASSWORD. Bring-up exits without it |
| The bootstrap token | From one of the sources above |
| Network | None. A cell needs no egress to start, licence itself, or enforce policy |
Boot it
The image runs the orchestrator as a service. The explicit form is:
python main.py start # also: stop | restart
start is single-instance guarded: a second concurrent start fails loudly with
the remedy rather than contending for the same ports.
On first boot a startup controller runs before any service launches, detects that a licence is present and no identity is on disk, and drives the full setup flow: licence, organization, node, administrator, cell, crypto, fabric. See the quickstart for the full decision tree.
Complete setup
The console's setup wizard is the normal route, and it will ask for the bootstrap token. To drive the whole flow headlessly instead, post to the bootstrap endpoint with the token.
Setup is durable and ordered: interrupted, it resumes rather than restarting. When it reports done, the cell is usable immediately with no second configuration pass.
Until it completes, every authenticated route returns 503. That is the setup
gate, not an outage.
Scenario 2: More cells under one node
The first cell mints the node. Every later cell enrols under it, and peers recognise node identity rather than cell identity.
There are two join paths. Pick by how the cell is provisioned.
Path A: join at boot (the image path)
Set S9_JOIN_ENDPOINT on the joining cell before first boot. The startup
controller detects it, takes the join path instead of the mint path, and the cell
attaches to the existing node.
This is the right path for a cloud image, an autoscaling group, or anything where the cell is configured before it starts.
Path B: join from a staged bundle
Where a cell is already running, or where the join material arrives separately, an operator mints a signed join-authorization bundle from the node and stages it on the joining host, by cloud-init, removable media or copy.
# On the joining cell
curl http://localhost:8080/api/public/setup/bundle/detect
curl -X POST http://localhost:8080/api/public/setup/bundle/join
The server verifies the bundle's signature and runs the cell-join ceremony. See joining the fabric.
Place the cells and wire replication
s9-admin topology list-zones
s9-admin topology assign <cell_id> <zone_id> --rack <rack_id>
s9-admin topology tree
Zone and rack are a hard placement constraint the platform honours before cost. Replication is declared per table rather than switched on globally; see platform replication.
Scenario 3: Sovereign and air-gapped
Air-gapped bring-up is the same entrypoint and the same setup flow. Nothing on the critical path needs a round trip: licence verification, signature checking and classification all evaluate locally against a build-embedded key.
What changes:
- The licence arrives as a file, not a fetch.
- The bootstrap token arrives with the image or on media, never from the network.
- Further cells join by Path B, with the bundle delivered on removable media and verified against a pre-staged trust anchor.
- Set the posture to sovereign or tactical. Under tactical, outbound egress fails closed and classification red lines are compiled in.
For the device half, see operate a device air-gapped.
Scenario 4: Cloud-enabled
Cloud is a configuration and connectivity posture, not a different entrypoint. Same image, same setup flow, with a cloud storage backend selected: a provider, and either S3-compatible object storage through an endpoint override (which also covers MinIO and Ceph) or Azure Blob with a key vault for secrets.
Online key provisioning and relay become available and remain conveniences, not requirements. The same cell keeps running if the link drops.
Running from source, for development only
This section is for working on the platform itself. A customer deployment never does this. The product ships as a sealed, licensed machine image; see the quickstart. If you are standing up a node to use it, use the scenarios above.
From a checked-out repository with the Python and Node toolchains installed:
npm run dev
which wraps:
cd backend && PYTHONUNBUFFERED=1 \
SYSTEM9_CONFIG_PATH=../config/config.yaml \
PROJECT_ROOT=.. \
python main.py
SYSTEM9_CONFIG_PATH points at the layered config root; PROJECT_ROOT resolves
data paths. A development build boots unlicensed; a production image with no
licence and no join endpoint logs the remedy and exits.
The front door comes up on :8080, the internal control plane on :9900, and the
console dev servers on :5173 and :5174. Public TLS is off in development.
Where a cell keeps its state
Local SQLite databases under ${DATA_DIR}/${ENV}/sqlite/<name>/<name>.db:
system, directory, sessions, messages, catalog, data, storage,
compute and the event lake. Each maps to one owning service. There is no
migration tool: schema is established by the services and the setup flow.
Verify it is up
Work down this list. Each step tells you which of the previous ones succeeded.
| # | Check | How |
|---|---|---|
| 1 | The orchestrator is running | python main.py start is refused by the single-instance guard |
| 2 | The front door answers | An HTTP request to :8080 (or :8443) is served |
| 3 | Setup finished | Authenticated routes stop returning 503 |
| 4 | The node exists | s9-admin topology tree prints the node |
| 5 | Every cell attached | The same tree lists each cell under it, in its assigned zone and rack |
For a multi-cell node, step 5 is the real one. A cell that booted but did not join shows up as its own node, not as a cell under yours, which is the failure worth catching before you put work on it.
When it does not come up
| Symptom | Most likely cause | What to do |
|---|---|---|
| Setup is refused | No bootstrap token, or the wrong one | Read it from the startup banner or config/bootstrap_token. Nothing issues it over the network |
| The cell exits at boot with a licence message | No licence on a production image | Supply S9_LICENSE_KEY or place the file at config/license.s9lic |
| The cell exits asking for a password | S9_ADMIN_PASSWORD is unset | Set it. This is deliberate: a cell that minted an unreachable administrator would be an estate nobody owns |
Authenticated routes return 503 | Setup has not completed | Finish the setup flow. This is the gate, not a fault |
| A second start fails | One is already running | Expected. Use restart, or stop the first |
| Setup reports already provisioned | The node is initialized | Setup cannot be re-run. Reuse the cell or start from a clean image |
| A joining cell minted its own node | Neither join path was taken | It booted without S9_JOIN_ENDPOINT and found no bundle. Reprovision it; a minted node cannot be retrofitted into another |
| The join bundle is rejected | Signature or trust anchor mismatch | Re-mint the bundle from the node. An air-gapped cell needs its trust anchor pre-staged |
Across the deployment matrix
| Corner | First boot | Further cells |
|---|---|---|
| Single-cell, cloud | License-driven boot; the small-site and dev default | |
| Multi-cell, cloud | First cell mints the node | Enroll via join bundle over the network |
| Single-cell, air-gapped | Signed-bundle boot, verified locally | |
| Multi-cell, air-gapped | Signed-bundle boot | Enroll via join bundles on removable media |
Related
- Initialization & setup, the phase-gated setup flow and cell profiles in depth.
- Joining the fabric, what a fresh cell does when it enrolls under a node.
- The cell model, the instance model behind single- and multi-cell.
- Choose a deployment: posture, scale, connectivity, and edge profile before you build.
- Ports & protocols reference, every listener a running node exposes.