Unstructured and media ingestion
The compliance answer that covers your databases and excludes the file shares, the mailboxes, the scanned documents and the recordings is covering the easy half. The exposure usually lives in the other one.
This page is how the other half enters the platform.
The chain
Figure UM1: two outcomes, kept distinct. A discovered file always becomes a catalog asset with its format, size, type and content hash. Its transcript, entities and detections are produced only when the extraction gate allows. With the gate off the media is catalogued and nothing is inferred, which costs nothing and reveals nothing.
Discover
| Source | What is found |
|---|---|
| File shares | SMB, NFS and local paths |
| Object stores | Buckets and prefixes, with format detection |
| Office documents | Spreadsheet sheets, tables and named ranges; document body tables; presentation slides |
| Content platforms | SharePoint, OneDrive and mail, with delta-token cursoring so a rescan reads changes rather than everything |
| Mailboxes | Archive files and live mail |
| Medical imaging | DICOM |
| Imagery and video | NITF, full-motion video with embedded metadata |
| Source code | Repository contents |
Parse
Parsing is per format and bounded. Structured definitions, schema files and delimited data each get a real parser, and a per-file byte cap limits what any one parser ingests.
Where a format is not parsed, it is surfaced as a file with its size and path rather than being silently skipped or guessed at.
Documents become a retrievable corpus
Parsing tells you what a file contains. Document ingestion turns it into something an agent or a query can retrieve against: a raw file goes through detect format → extract text → chunk, and the chunks are persisted as that tenant's corpus.
| Ingest | POST /v1/document-ingest/ingest |
| Read back a document's chunks | GET /v1/document-ingest/documents/{document_id}/chunks |
| Formats | PDF, DOCX, TXT, CSV, JSON, HTML, Markdown, XML |
| Options | Chunk size and overlap, table extraction, OCR, PII detection, and a classification level, each per ingest |
This is a different thing from the JSON document database the platform also offers, which happens to share the word. Here the input is a raw file and the output is text chunks.
One corpus, one tenant, and no way to blur that
The corpus belongs to a workspace or a clean room, never both and never neither. That exclusivity is not a convention applied at one layer and hoped for elsewhere: the request validates it, the route enforces it, and the backing store constrains it at the schema.
Reading and writing are governed separately, and neither reduces to a flat capability:
- Ingesting into a workspace needs the data-write permission.
- Ingesting into a clean room needs that and the ability to manage the room, so holding a broad write capability does not let you push a corpus into a room you are not a steward of.
- Reading chunks is confined to a tenant the caller administers. A general read permission is not enough on its own, because it would otherwise disclose any tenant's corpus.
The refusal shape on the read path is the detail worth keeping: an unauthorized read returns an empty list, not a permission error. A "you may not see this" response would confirm that a corpus exists for that tenant, which is a fact worth protecting on its own. Deny-closed here means telling you nothing, rather than telling you that there is something you cannot have.
Understand
Audio transcription
Transcription is a real capability, and where it runs is the thing to get right.
Transcription runs in two places, and both are opt-in.
On an Edge Agent. A Whisper-class engine is compiled into the shipped binary and runs on the device on CPU. The model is separate: it arrives as a signed bundle applied to the device's model registry, and without one the tool refuses with "no speech model provisioned" rather than inventing a transcript. With the engine compiled out, it refuses with "engine not built". Neither failure fabricates.
Input is a WAV file, or raw little-endian 16-bit PCM with a declared schema, supplied from the device's workspace. The result is recorded as a device-signed, tamper-evident perception record and synced to the cell's knowledge plane, best-effort by contract: on an unenrolled device, or if the local store cannot be opened, the transcript is still returned to the caller, unsigned. The signature is an addition, never a gate.
On the cell. The speech provider is resolved at startup, and the speech library is an optional extra alongside the one for text recognition. It stays off until a deployment opts in, and it is honest in both directions:
| Condition | Result |
|---|---|
| A model is configured and the speech extra is installed | Transcription runs on the cell, on CPU, with no egress |
| No model configured | The capability reports itself unavailable |
| Model configured, library absent | Fail-closed and graceful: the cell still boots, the capability reports no provider |
Neither failure fabricates text. Running on CPU with no outbound call is what makes it usable on a sovereign or air-gapped cell at all.
How the cell-side path behaves:
- No external API call. Audio would be transcribed on the cell, which for an air-gapped or classified deployment is the difference between usable and prohibited.
- No media toolchain. Audio is interpreted as 16 kHz mono PCM in process, so the deployment carries no codec dependency. The corollary is that a caller must supply decoded audio: there is no decoder on this path, so a compressed container is not transcribable as-is.
- The transcript becomes a signed observation bound to the content hash of the exact bytes the extractor saw, written marked unsigned rather than carrying a fabricated signature when no key is available.
- A missing engine is reported, not hidden. The record says transcription was not attempted rather than returning an empty transcript as a result.
Two limits to know even once it is composed: there is no speaker diarization (the configuration flags exist and nothing reads them), and segment timings are produced by the engine and then discarded when the transcript is stored, so a stored transcript is one block rather than a timed track.
Entities: ontology terms, grounded in evidence
Entity extraction from text runs today, and it is important to be exact about what it does, because the obvious reading is wrong.
It is ontology-term recognition, not named-entity recognition. The matcher takes the object types in your workspace's own ontology, together with their aliases, and finds every occurrence of those surface forms in the text. It will not discover "Acme Corp" in a document unless someone authored "Acme Corp" as an alias of an object type. What you get out is bounded by what you modelled going in.
That is a narrower claim than "we find the people and places in your documents", and it is the honest one. It is also a deliberate design: the ontology is governed, versioned and hash-chained, so extraction cannot invent a category nobody approved.
What happens to a match is the part worth the attention:
| Stage | What it produces |
|---|---|
| Match | An occurrence of an ontology term, with the span it was found at |
| Observation | A record identified by the SHA-256 of the submitted content, so a resubmission is deterministic, Ed25519-signed and hash-chained |
| Candidate | A pre-fusion extraction, held local to the cell as working state |
| Canonical | A bitemporal entity version, written only after deduplication and a promotion gate |
Duplicates are resolved by exact match, fuzzy match at a 0.85 threshold, or vector similarity at 0.90, grouped with union-find. Evidence is graded on a four-rung ladder from unverified hearsay through corroborated and attested to documented. A claim marked as real cannot be constructed without evidence: the type refuses it.
The promotion worker runs by default on a one-minute cadence, and canonical entities converge across the organization's nodes while candidates stay local. A cell with no signing key refuses to write canonical entities rather than persisting unsigned ones.
Grounding is document-level, not sentence-level. The span is computed, validated, and returned by the preview endpoint, then dropped before persistence. A canonical entity traces to which document, not which sentence.
Confidence is coarse. The live path assigns a fixed score, which maps to the middle of three buckets, so everything it extracts lands as mixed rather than carrying a model's own score.
Open-vocabulary extraction depends on a permitted model provider. The engine, with its cost gates, circuit breakers and coverage checks, is assembled at cell startup and its scan endpoint serves. On an air-gapped cell it resolves the local model registry rather than reaching out. Where deployment policy permits no model provider at all, the endpoint refuses rather than running half-built.
Structured data gets a different mechanism
Entity detection over tables does not use a language model, and should not.
Column values resolve through deterministic resolvers for emails, hosts, customer identifiers and parties. Sensitive-column typing runs on name and value patterns. Identity is reconciled across objects sharing a strong signal such as an email or an employee identifier.
A cross-source resolver then bridges the two worlds by schema fingerprint, matching a document-derived object to a catalog table with a compatible shape: this quarterly report has the same shape as your quarterly orders table.
Language models for prose, deterministic signals for tables, one canonical entity underneath.
Video and audio formats
Audio is detected by content rather than by file extension, covering the common uncompressed and compressed formats. Video covers the common containers, and the extractor does not demux by itself: the container backend is supplied, so the platform loads cleanly whether or not a media toolchain is installed, and video composes the image path over sampled keyframes with the audio path over the soundtrack.
Analysis results carry detections, extracted text, the transcript, a scene description, classification tags, an embedding and a confidence, each labelled with the provider that produced it.
Where content carries content-provenance credentials, analyzers verify them and attach the verdict to the result rather than assuming the media is what it claims.
The gate
Extraction is off by default and fail-closed. Turning it on is a governed decision, evaluated against domain policy, the cell's profile, an authorization check and the skill catalog.
The reasoning: cataloguing a file costs storage, and understanding it costs inference. Those are different decisions with different budgets and different disclosure consequences, so they are separate switches.
Non-document objects are never routed to extraction regardless of the gate, and a source is never silently dropped: it goes to extraction or it goes to plain ingestion.
What comes out
| Outcome | When |
|---|---|
| A catalog asset with format, size, content type, modification time and content hash | Always |
| A signed observation carrying the transcript, entities or detections | Only when the gate allows |
Observations are readable only within their scope, retrievable through a governed route that fails closed on an unbound or cross-scope read rather than serving anything. They feed retrieval, the property graph, and the erasure path that removes them on request.
The transcription engine, the signed sink and the governed route are each built and correct, and the final production composition that joins them is owed work. Discovery, cataloguing and the structured-entity path are unaffected.
Related topics
- The whole arc this page sits inside: from capture to decision.
- Every format read and identified: supported file types.
- What connects the sources: connectors and ingestion.
- Scanning an estate rather than ingesting it: Discovery and Mapping.
- Where extracted entities are promoted to canonical: Surveillance.
- Devices and sensors, a separate family: sensor intake.