Data Modeling
Data Modeling is the transformation-project layer of the Data Platform: the place you declare how a raw table becomes a curated one. It uses a vocabulary analysts already know (projects, models, and seeds) and compiles them into a dependency graph that runs inside the governed engine, not beside it.
The building blocks
| Object | What it is |
|---|---|
| Project | The unit of organization for a set of related models and seeds, scoped to a workspace. |
| Model | A transformation defined as a query: the definition of a derived table, with its dependencies on upstream sources and other models. |
| Seed | A small, version-controlled dataset loaded as a table, so reference data lives with the project rather than as an untracked import. |
| Run | A materialization of the project's graph: each model and each model run is recorded, so a build is an auditable event, not a side effect. |
A model compiles into a governed DAG
Models declare their dependencies rather than their order. A compiler resolves those dependencies into a directed acyclic graph, and an executor materializes it, running each model's SQL against the platform's own engine.
Because the build runs on that engine, there is no unpoliced back door to the data: every read a model performs passes the same in-engine enforcement, row filters and column masks compiled into the plan, as any interactive query. A model cannot see what its author could not see.
Figure: seeds and catalogued sources feed the models of a project; the compiler resolves dependencies into a DAG, and a governed run materializes catalogued assets with lineage captured. The run applies the same in-engine policy as any other read, the modeling layer is not a privileged path.
Table equivalent
| From | Relationship | To | State (meaning) |
|---|---|---|---|
| Seed (reference data) | feeds | Model A (staging) | candidate / in-flight (cyan) |
| Catalogued sources | feed | Model A (staging) | neutral input |
| Model A | is depended on by | Model B (curated) | candidate / in-flight (cyan) |
| Model B | compiles into | DAG | neutral |
| DAG | runs under | Governed run (in-engine policy) | verified (green) |
| Governed run | produces | Materialized asset + lineage | verified (green) |
Modeling beyond transformations
The modeling story does not stop at transformation projects:
- Dimensional modeling. Star-schema modeling over catalog assets, with slowly-changing dimensions (types 1, 2, and 3), conformed dimensions shared across subject areas, grain enforcement, an aggregate navigator that routes a query to the most appropriate aggregate level, and incremental refresh. A dimensional model materializes into real tables, covered below.
- Semantic modeling. Reusable metric definitions, covered below.
Semantic modeling
Reusable metric definitions: measures, dimensions, filters, queried consistently across tools, with hierarchy resolution for drill paths. Gluing models across cells that cover one domain is Federated Semantics, which exchanges signed projections rather than raw data.
A dimensional model becomes real tables
Designing a star schema is the easy half. The half that decides whether a dimensional layer is real is loading it correctly, and that is what the materializer does.
A model materializes into versioned Delta or Iceberg tables on the lakehouse. The format is resolved by policy rather than hardcoded, so an estate that has standardised on one does not get the other.
The load order is the part worth knowing, because each step exists to prevent a specific way dimensional loads go wrong:
| Step | What it does |
|---|---|
| Grain gate | Refuses a load whose grain does not match the model's declared grain, before anything is written |
| Dimension load | Type 2 expires the current row by merging on the surrogate key, flipping its current flag and expiry date, then appends the new version. Type 1 updates in place |
| Key mapping | Builds the natural-key to surrogate-key map the facts will need |
| Fact load | Resolves foreign keys through that map, then appends |
Type 2 done as an expire-then-append merge is the difference between a dimension that carries real history and one that quietly overwrites it.
Scoped by database, not by label
The materializer keys on the database namespace, not on a workspace name.
A workspace and a clean room each own their own databases, and both reach the lakehouse through the same namespacing. So a clean-room load lands only in that room's namespace, with no cross-scope bleed, by construction rather than by a check that could be forgotten. The caller resolves which database; the runtime cannot write outside it.
Materialized tables register in the catalog and carry the same audit, activity record and metering as any other lakehouse write. A dimensional load is not a side door around governance.
Across the deployment matrix
- Single-cell: projects, models, and seeds live in the cell; a run materializes into that cell's catalog.
- Multi-cell: a workspace's models span cells; federated semantics glue models across cells covering one domain without moving raw data.
- Sovereign / air-gapped: the compiler and executor run on the cell with no external build service in the path.
- Cloud-enabled: the same modeling layer, with federation to foreign catalogs available as an option.
Related
- Data products, package a curated model as a product with a contract.
- The catalog and lineage: where models, lineage, and semantic definitions are recorded.
- Governed reads, the policy a model's every read inherits.
- In-engine enforcement, why the build path is not a bypass.