Skip to main content

Chat & LLM

Chat is the conversational surface over the AI system. A chat turn is a reason → optional tool calls → reflect loop, bounded by an iteration budget, and the same role / content / tool-call / tool-result protocol drives it on both runtimes, so a turn round-trips through the fabric without translation. What governs the turn is identical whichever runtime serves it; only which model and how far a request may travel change.

Date the capability

Both the chat tool surface and sovereign local inference were off in every deployment until the release that fixed a configuration-prefix mismatch, not disabled by policy, but unreachable by accident. Before quoting either capability, confirm the version. The lesson is the quotable part: a feature can be built, tested, and switched on in configuration and still be off everywhere, because nothing tested that the key the code reads is the key the tree holds.

Providers: hosted, local, or none​

A provider abstraction sits under chat with adapters for hosted providers and self-hosted local runtimes, a registry with failover, and per-agent, per-session and per-organization token accounting. Which providers are reachable is decided by the deployment tier, not by the chat surface:

TierProviders reachable from chat
AI off (default)None: chat over an LLM is not available
SovereignLocal models only, served in-process; no vendor egress
FullLocal models and third-party providers, through a secure provider that enforces classification policy and redacts before anything leaves the cell

The choice is enforceable per workspace, per classification, per cell, a customer can use a frontier model where policy permits and drop to local where it does not, without changing how anything is metered, governed, or audited.

How a chat call is governed, end to end​

Every chat turn passes the same ordered guards before a model ever runs, and a refusable gate before any tool it requests takes effect.

*Figure: a governed chat turn. Untrusted content is delimited and validated (amber) on the way in; an inference-egress residency check refuses closed before any context leaves the boundary; the reasoner routes to a local model (green) for sovereign, offline, or classified work and relays (cyan) only where posture and egress permit; output passes a redaction intercept; and any tool call the model requests must clear a net-potential-gain gate, only NET_POSITIVE proceeds, and uncertainty refuses. Ceiling: the NPG gate is a refusal gate only, no behavioural / alignment score is produced. Redaction on the completion path is a core Model Serving property; the edge content guard does not do general PII/secret redaction.

Table equivalent

StepWhat happensOutcome on failure
Content guard INUntrusted spans (tool output, recalled memory, peer messages, attachments) are delimited; the prompt is validated against injection patternsBlocked content is withheld and replaced with a guardrail marker
Inference-egress residencyThe residency of the workspace / clean room is resolved live before any context is bound or sentRefuse closed: an unsatisfiable pin, or a read fault, refuses rather than originating in the wrong place
Route the reasonerChoose local vs relay by posture (sovereign / AI-off force local), classification, and connectivityA relay failure degrades to local; a local failure never escalates to a relay
Output interceptClassification-aware redaction on the core serving path (read-down model)Output is redacted per classification
Per-tool NPG gateEach requested tool call is scored for net effectOnly NET_POSITIVE proceeds; NEGATIVE, NEUTRAL, and INSUFFICIENT_DATA all refuse
Turn completeTokens accounted, action attributed, decision audited

Routing: sovereign-degrade, never sovereign-escalate​

The reasoner selection is deliberately asymmetric. Sovereign or AI-off posture, a classified-egress gate, and being offline all force the turn to a local model. When a relay is chosen and fails or is absent, the turn degrades to the local model, but a local failure is never allowed to trigger a relay, because a local failure must never cause unexpected egress. Each round-trip emits one reasoning span recording which runtime served it and whether it degraded.

At the edge this is a confidence-gated cascade: the cheapest sufficient local tier answers first, and escalation is confidence-gated, and even then the leaf never calls the frontier itself, it flags a partial for the conductor to decide. See the edge path.

Chat is scoped to a boundary​

This is the structural fact everything else about chat follows from, and it is easy to miss because most assistants are a sidebar rather than a scoped surface.

A chat session runs as a chat principal inside a workspace or a clean room. It is not a service account with broad read access answering on the user's behalf. It reaches data through the workspace access path, which means:

The assistant cannot see what the asker cannot see.

That single property is what makes an assistant safe to point at governed data. The alternative, an assistant with its own elevated access that filters results afterward, is the shape that leaks: the model has already read the thing it is filtering.

The same surface, three places​

WhereWhat it is
A workspaceThe assistant over that workspace's own data, members and grants
A clean roomThe room's assistant, one of the four always-on tabs precisely because it is a room guarantee, scoped to the room and its egress policy
An Edge AgentA conversational surface and a single-shot goal surface on the device, reasoning on a local model with no egress

The clean-room case is the interesting one. The room's assistant cannot become a side channel that answers across the trust boundary, because it is bound by the same aliasing and the same egress gate as anything else in the room. A persona switch cannot take it off screen either, since it is always-on by design.

Progressive tool disclosure​

A workspace console carries a very wide capability surface, and putting every tool schema into every prompt would destroy both the token budget and the model's ability to choose correctly. Access is not competence.

So the prompt carries a compact manifest of capability families scoped to the workspace's grants, and the agent pulls a full schema only when it needs one.

The scoping matters as much as the compression: the manifest reflects what this workspace actually holds, so the assistant is not told about capabilities it could not use even if it chose them.

Working memory and history​

A scoped "remember this" note the user pins is formatted into later turns, and long sessions get managed history compaction rather than an unbounded context or an abrupt truncation.

Tools and grounding​

Chat is a tool-using surface: the model can call workspace tools, an external tool bridge, and progressively-disclosed capabilities narrowed by a tool-search meta-tool. Two disciplines matter for chat specifically:

  • The model supplies intent, never authority. A capability is never granted by prose in a system prompt, every capability decision is taken by the permission layer. A briefing is a floor, not a ceiling.
  • Every tool result is untrusted until delimited. Tool output is the classic injection vector, so it passes the content guard before the model sees it. See AI / LLM security.

Across the matrix​

  • Single-cell: chat runs entirely on one cell; residency is trivially satisfied and a lone sovereign device reasons locally.
  • Multi-cell: the residency check resolves against live fabric membership and originates the call on the residency-correct cell.
  • Sovereign / air-gapped: chat runs on local models only; a relay is unreachable by construction, and the turn answers at best-available.
  • Core vs edge: the same loop and protocol; the edge adds the confidence-gated cascade and answers locally by default.

Run it at the edge​

On the Edge Agent, the same protocol is driven by s9 chat (interactive or one-shot) and s9 do (an autonomous mission). Both reason locally, or relay to the fabric with --relay when connected:

# One-shot question, grounded in a workspace over relay, with a durable audit trail
s9 chat "which models are loaded on this device?" --relay --workspace ws-acme --audit

# An autonomous mission that pauses consequential actions for a human decision
s9 do "triage the failing sensor and draft a report" --hitl

--audit records every reasoning step to a device-signed, hash-chained journal; --hitl writes each consequential tool call to a durable approval queue instead of running it (resolve it with s9 hitl). See the tutorial Enroll a device and operate it air-gapped for the full flow, and the on-device agent reference for every flag.

On-device reasoning needs a signed model bundle

The default Edge Agent build compiles in the real on-device inference engine; what it needs before it can reason locally is a signed model bundle to import. A minimal build (--no-default-features, for a sensor-class device) links no engine and falls back to a stub runtime. The governance around the turn, the tool gate, the audit trail, the HITL queue, is real regardless of the runtime.

Verifiable answers​

Every answer can return a signed verification bundle proving why the answer is what it is and that no classification marking was bypassed, with receipts hash-chained per session. See Prove-It.