AI functions
An AI function is a function that invokes a model. It is not a separate subsystem bolted onto the data platform. It is one of five call sites of the single function object, reachable behind the same sandbox, the same lifecycle, and the same metering as every other function. That is the whole idea worth holding: the thing that calls a model is governed exactly like the thing that masks a column, because they are the same object invoked from a different place.
This page covers what an AI function is, the AI-in-SQL surface it exposes, how it composes with the five computed isolation tiers, and how a model-invoking function is governed at call time. Two framing facts apply, from the AI system: AI is off by default, and the inference an AI function reaches is sovereign-first, a local model, resolved before any vendor, with no egress unless a deployment deliberately opts up.
One function object, five call sites
A function is registered once and invoked through one contract, a function identifier and a batch of values in, outputs out, and that single invoker body fans out to the right runtime by the function's declared kind. Five places reach it, and an AI function is the third:
*Figure: one function object, five call sites. Every call site reaches the same invoker behind the same sandbox; the AI call site (cyan) additionally resolves a model through the governed, sovereign-first inference bridge. Ceiling: the five call sites are real, and the peer-dispatch path is driven from both ends. The sender is assembled in the platform bootstrap with an explicitly-constructed service discovery rather than resolving its transport from shared runtime state.
Table equivalent
| # | Call site | What the function does there |
|---|---|---|
| 1 | ETL transform step | Transforms a column inside a pipeline, vectorized over values |
| 2 | Column mask | Bound by data-security policy as a column mask; fails closed to a redaction on error |
| 3 | AI function | Invokes a model through the governed inference bridge |
| 4 | Agent function | A function an agent may call: run as a leaf, so it cannot itself fan out |
| 5 | Peer-cell dispatch | Runs on the cell where the data lives; the peer re-runs the same enforcement |
The lifecycle around the object is catalog-managed and the same for all five: register → version immutably → publish along a visibility ladder → bind to a route → assign masks and tags → resolve at invocation, with per-invocation metering that carries the isolation tier as an attribute into the tamper-evident ledger and the cost record.
What a function declares
Two axes, deliberately orthogonal: what the body is, and where it is allowed to run.
| Axis | Values |
|---|---|
| Runtime kind | Declarative, SQL, code, AI, or agent |
| Function kind | Mask, row filter, transform, endpoint, application logic, or model |
| Invocation shape | Column mask, row filter, transform, stream, endpoint, event, step, or schedule |
| Locality | Follow the data, owner only, service group, node local, or anywhere |
An AI function declares no code at all. Its body is a governed prompt template with restricted field substitution: no attribute access, no format specifiers. An AI function cannot reach into object internals, because it cannot express that.
An agent function runs as a leaf: its spawn depth and fan-out are both zero, so a function that invokes an agent cannot become the root of a swarm.
The registration guardrail worth knowing
A data-plane code function that cannot be pushed down into the engine must declare a batch signature. A per-row sandboxed scalar is refused at registration rather than accepted and discovered to be slow in production.
A pushdown analyser marks the expressions that can run entirely in-engine, which cost zero sandbox crossings.
Masks, and the two properties that matter
A mask is a function bound to a column. Two behaviours are worth stating because both are easy to get wrong:
Removing a mask does not reveal the column. The column stays marked sensitive, so it falls through to deny-by-default redaction rather than becoming readable. Unbinding a mask is not an exposure event.
Reassigning a mask is atomic. The swap is a single update, so there is no window in which the column is unmasked between the old binding and the new one.
Versioning, publishing and routing
Versions coexist, and a published version's definition is frozen permanently. Only its visibility moves along the ladder from draft through to retired, and the ladder runs one way.
Rolling back therefore means deprecating the current version and re-publishing the prior one. The definition of a published function is never edited in place, so what ran last Tuesday is still exactly what you can read today.
Before publishing, a function can be dry-run against your own sample rows in the isolation tier it would really use, producing a before-and-after report. Nothing is recorded to any ledger by a dry run.
A route maps a public name to a primary version and, optionally, a candidate. Canary selection uses a deterministic hash of the caller, so the same principal consistently reaches the same version. A per-request coin flip would make a canary impossible to debug.
The AI functions, by name
Inference is reachable from SQL, by the analysts who own the data. Here is what ships.
Row functions
Each is called per row, but not executed per row. See why that matters below.
| Function | What it does |
|---|---|
AI.COMPLETE(prompt[, model]) | Free-form completion, where the column value is the prompt |
AI.CLASSIFY(text, 'label_a', 'label_b', …) | Single-label classification against labels you supply inline |
AI.EXTRACT(text, schema) | Structured field extraction, returning a JSON object |
AI.SUMMARIZE(text) | A one-sentence summary |
AI.SENTIMENT(text) | One word: positive, negative or neutral |
AI.TRANSLATE(text, language) | Translation to the named language |
AI.EMBED(text[, model]) | An embedding vector, routed to the embedding model rather than the completion model |
Filtering with a model
AI.FILTER(text, 'predicate') is a model-as-judge boolean, usable in a WHERE
clause.
The implementation detail worth knowing is that the filter is applied first, so model spend only lands on the rows that survive it. Ordering it the other way would be the expensive mistake.
Aggregates
| Function | What it does |
|---|---|
AI.SUMMARIZE_AGG(column) | One summary across the whole column |
AI.AGG(column, 'instruction') | One reduction across the column, under your instruction |
These are a single call over the group, not a call per row.
Retrieval
AI.SEARCH(space, 'a question in plain language'[, top_k][, ground]) does
embedding, nearest-neighbour retrieval and, optionally, grounds an answer in what
it found. It returns the answer alongside the supporting hits, so the citation is
part of the result rather than something to reconstruct.
Vector functions
| Function | What it does |
|---|---|
VECTOR.SIMILARITY(a, b[, metric]) | Cosine, Euclidean or dot-product similarity. The one function here that runs natively in the engine with no model call at all |
VECTOR.EMBED(text[, model]) | Governed embedding |
VECTOR.SEARCH(space, query[, top_k][, filter]) | Nearest-neighbour search over a real indexed vector space |
Machine learning in SQL
Not language models, but customers reasonably call these AI functions too.
Scoring a registered model:
| Function | What it does |
|---|---|
ML.PREDICT(model, …features) | Row-level scoring. Model reference resolves by name, alias or explicit version |
ML.PREDICT_TABLE(model, TABLE source) | Table-level scoring, returning the source columns plus a prediction and per-class probabilities |
No model required. These fit on the spot:
| Function | What it does |
|---|---|
ML.FORECAST(source, ts, target, horizon[, frequency]) | Time-series forecasting |
ML.DETECT_ANOMALIES(source, target[, contamination]) | Isolation-forest anomaly detection |
ML.CLUSTER(source, k[, features]) | K-means clustering |
Forecasting is worth a note on honesty: it falls through a chain of backends, preferring specialised forecasting libraries and falling back through statistical models to a deterministic seasonal or linear method, each producing prediction intervals derived from real residuals. The result reports which backend actually ran, so a fallback is visible rather than silently worse.
Evaluating and explaining:
ML.EVALUATE, ML.CONFUSION_MATRIX, ML.ROC_CURVE, ML.FEATURE_IMPORTANCE
(global, by permutation) and ML.EXPLAIN_PREDICT (local, for one prediction).
Managing models: CREATE MODEL, ALTER MODEL and DROP MODEL as ordinary
data-definition statements.
There is no AI.MASK, AI.GENERATE, AI.FORECAST or AI.SIMILARITY. Masking is
a function object bound to a column, not
an inline call. Forecasting is ML.FORECAST. Similarity is VECTOR.SIMILARITY.
There is also no reranking function and no general recommender. The knowledge layer declines reranking deliberately, on the grounds that the caller should re-rank by their own signals rather than inherit hidden ranking.
The statement is intercepted, not the row
This is the most important design decision on the page, and it is the one that separates a governed AI SQL surface from an expensive one.
An AI function is never bound as a per-row user-defined function. The whole statement is intercepted before it runs, so the cost governor sees the query at planning time.
The alternative, binding AI.COMPLETE as an ordinary scalar function, is what
turns a careless SELECT over a large table into a very large bill that nobody
can intercept, because by the time the first row is evaluated the query is already
running. The codebase names this the footgun it is avoiding.
Intercepting the statement gives three things that per-row binding cannot:
A cost estimate before execution. Rows times estimated tokens times price, with the row count read through a cheap count on the same governed path.
Deduplication. One model call per unique input, not per row. A column with ten thousand rows and forty distinct values costs forty calls.
Batching. Above a row threshold, calls are forced into batches rather than issued one at a time.
The cost gate refuses, it does not throttle
The governor runs estimate, then model allow-list, then admit. Exceeding the budget is a typed refusal, not a queue and not a slow lane.
Two details make the budget behave sensibly in a real deployment:
The ceiling is apportioned across cells. A single global per-workspace ceiling would be roughly N times wrong across N cells, and asking every cell for consensus on each call would be too slow to sit in a query path. Each cell holds a sub-budget and fails closed when it is exhausted.
A failed statement settles to zero. A query that errors does not permanently consume the budget it reserved.
A model allow-list can narrow which models a workspace may reach, and narrowing is the only direction it travels.
Where an AI function can be called
The same governed path serves every one of these:
| From | Notes |
|---|---|
| Analytical SQL | The primary surface |
| Lakehouse SQL | Over open-format tables |
| Streaming SQL | AI.* works on a live stream, not only a table |
| Transactional SQL | The same functions over the row store |
| Notebooks | Through the same SQL path, so policy is identical |
| Pipelines | Declarative enrichment steps in a transform |
| The document store | A projection operator on a document query |
| Create-table-as-select | The AI select is evaluated and rewritten into a plain statement, so the resulting table holds values rather than a re-running function |
| An HTTP endpoint | A published function invoked as a service |
| Agents | Through the tool surface |
The inference path
Figure AF1: one path, whether the caller is SQL, a pipeline, an endpoint or an
agent. The local model is asked first; a cloud provider is reached only when a
call names one explicitly and the deployment permits it. So a bare AI.COMPLETE
runs air-gapped.
Classification is read from the query, not asserted
Before anything runs, the source statement is parsed and the classification taken as the maximum across every table it references.
A statement that cannot be parsed, a table that cannot be resolved, or a table carrying no marking all resolve to a confidential baseline. The unknown cases fail toward protection rather than away from it.
What the egress gate decides
| Situation | Outcome |
|---|---|
| A sovereign local model | Always allowed |
| Inside a clean room | Any third-party egress is denied outright |
| Data at or above the confidential floor | Must stay sovereign |
| A model whose sovereignty is unknown | Treated as non-sovereign. Fail closed |
AI is off by default
The deployment profile ships as AI off. It is raised by explicit licence issuance and never lowered by oversight, and a sovereign or tactical posture can never acquire cloud inference.
With AI off, the provider is never registered, so the bridge raises and the surface reports that inference is unavailable. It does not write null enrichment into your table. Refusing is the correct outcome; quietly writing nothing would corrupt the result while looking like success.
The five isolation tiers
Every function runs behind a sandbox, and the sandbox is one of five tiers, and the required tier is computed, never chosen by the author.
*Figure: the isolation ladder, strongest on the right. The tier is resolved from code trust, data classification and cell regime; the runtime picks the lowest available backend at or above the required rank and fails closed rather than downgrading if none is available. Ceiling: the ladder and the resolution are implemented and enforced. The higher backends (microVM, quarantine, and even the OS and WebAssembly tiers) are capability-probed per host and are inert, the run fails closed, never downgrades, where a host is not equipped.
Table equivalent
| Tier | Boundary | Notes |
|---|---|---|
| T0 | In-process declarative: no arbitrary code | Declarative and SQL functions, enforced in-engine |
| T1 | OS sandbox | User-namespace isolation, syscall filtering, control groups, network namespace; no unsandboxed fallback |
| T2 | WebAssembly | Capability-gated and fuel-metered |
| T3 | microVM or userspace kernel | Probed against host virtualization |
| T4 | Quarantine cell | Placement isolation with no sensitive co-residency |
The tier is computed, and two rules make it fail-safe
The required tier is resolved from code trust × data classification × cell regime, and unknown inputs resolve to the stricter value, an unknown classification is treated as restricted, an unknown cell regime as the most isolated (air-gapped). Two rules then bind the result:
- Classification raises the tier. A published or externally-submitted function over restricted data is pushed toward quarantine, overriding data-gravity and locality. A first-party function over its own data gets no extra floor; the raise is reserved for the case that warrants it.
- The author may only tighten. An author's declared tier is combined most-restrictive-wins with the computed floor, an author may demand more isolation, never less. The runtime then re-pins the effective tier onto the execution context, so a backend can never run below what was required.
An AI function is code, so it never sits at T0, it earns at least an OS sandbox, and a published AI function over confidential or secret data is pushed up the ladder toward WebAssembly or quarantine by the classification rule, not by whoever wrote it.
The tier-raise mechanism is fully implemented and enforced. But on the shared invoker path that the ETL, column-mask, AI-function, peer and agent-tool call sites flow through, the data-classification axis is currently fixed rather than fed each invocation's live data sensitivity, so today the computed tier on that path is driven by code trust and cell regime, and the code trust is derived correctly from the function's real visibility. Feeding live per-invocation classification into the computation on that path is owed work. The rule is designed and enforced; the classification input is what is not yet fed on the shared path.
Governance at invocation
Two governance properties hold when a function runs, and both are resolved at call time, not declared up front:
- Clean-room egress is intersected at invocation. The invoker contract carries no room context. At call time the runtime resolves whether the function is bound into a clean room and, if so, intersects the function's declared egress with the room's egress policy, most-restrictive-wins, fail-closed. If the intersected posture permits no external destination, the runtime forces the sandbox to have no network at all, so the backend cannot egress even if the function declared it could. A lookup fault denies external egress rather than allowing it, and an externally-submitted function may not bind into a room.
- Metering carries the tier. Every invocation records the isolation tier it ran at into the tamper-evident ledger, the activity event, and the cost sample, so the isolation a model-invoking function actually received is auditable and billable, not merely configured.
The same sandbox model runs at the edge
The edge carries a mirror of the sandbox specification builder that turns a sub-agent's attenuated resource budget into a concrete, enforceable sandbox and the exact per-platform launch command. Four invariants are encoded structurally rather than configured:
- Resource limits derive from the grant: a sandbox can never request more CPU, memory or processes than its grant's slice.
- Bind-mount allowlist only: a read-only base filesystem, the entity's writable workspace, a temporary filesystem; no host home directory, no system configuration, no credentials, and no bind outside the allowlist is even representable.
- No ambient network: the namespace is network-isolated and the egress broker is bound as a socket file, not a route.
- No privileges: unprivileged user remapping, no capabilities, and a root-uid specification is rejected.
The edge builder emits real, enforceable launch commands on Linux (user-namespace isolation) and macOS (the OS sandbox profile). The Windows job-object / app-container surface is described in the design but is not implemented as enforcing code, on Windows the builder produces resource limits with no enforcing consumer. Treat Windows edge sandbox enforcement as owed work, not a shipped guard.
In the deployment matrix
The function object and its tier computation behave the same single-cell and multi-cell: a peer-dispatched function re-runs the same enforcement on the cell that holds the data, so distribution does not relax the sandbox. Sovereign and air-gapped, an AI function resolves a local model first and reaches no vendor, and an unknown cell regime resolves to the most isolated tier, the fail-safe default favors the corner the product is sold on. The one uneven surface is edge OS coverage, called out above.
Related topics
- The AI system (core + edge), the tier model (AI off by default), sovereign-first inference, and the governance an AI function inherits.
- Chat & LLM, the governed inference path an AI function shares.
- Data Pipelines, the ETL transform call site, and how a function step distributes across cells.
- compute service, where the function object, its call sites, and its sandboxes live.
- Data Governance, the column-mask call site and the enforcement plane a function runs behind.