Catalog interoperability
Most organizations already run a metadata tool. Replacing it is a project nobody asked for, so the catalog is readable by the tooling a customer already has through an OpenMetadata-compatible surface.
The interesting engineering here is not the format translation. It is that metadata discovery is itself an access-controlled operation, and the surface is built so that it never tells you about something you could not query.
Two faces, for two kinds of caller
| Face | For | Authentication |
|---|---|---|
| The compatible API | Internal and delegated callers, and metadata tooling inside the estate | An acting entity on the request, or a trusted system call over the internal transport |
| The external face | A metadata client outside the estate | A per-database credential over TLS, resolving to a directory entity |
What the compatible API exposes
Entity listings and lineage in OpenMetadata's shapes:
- Tables, topics and databases
- Tags and glossary terms
- Lineage, both for a specific entity and across the graph
- Search
A mapper converts catalog rows into OpenMetadata entity JSON, reading the real catalog chain rather than a parallel copy.
What the external face exposes
A deliberately narrower, read-only surface, because a catalog client discovers, it does not mutate:
- Catalogs on this node
- Schemas within a catalog
- Tables within a schema
- Table detail with columns
Enumeration is need-to-know enforced
This is the property worth understanding, and it is stronger than filtering results.
A caller must not even learn that a table exists if it is outside their scope or above their clearance.
Listings are filtered through the same visibility decision the rest of the catalog uses, and on the external face every table is gated through the same data-security orchestrator the engine adapters use. A denial does two things:
- It removes the table from listings, and
- It returns 404 on a direct fetch, rather than 403.
That distinction is the whole point. A 403 confirms the thing exists and you
cannot have it, which is itself a disclosure. A 404 does not.
Deny-closed, and audited
If the security plane is wired but an enforcement check errors, the table is dropped rather than leaked. The failure mode is invisibility, not exposure.
Every table enforcement is recorded through the shared auditor, so metadata reads are on the record like any other read.
The rule the surface is built around
The metadata surface's job is to never advertise a table the caller could not query.
This is metadata discovery only. Row-level data is consumed over the SQL and native engine faces, which apply their own row and column policy. Keeping the two consistent is what stops the catalog becoming a side channel that maps out data you have no access to.
Trusted system calls
A trusted internal call carries no acting entity, and is returned unfiltered. That is not a gap: on that path the transport is the boundary, since the caller has authenticated with a system token or mutual TLS on an internal interface. Filtering by an entity that is not present would be filtering by nothing.
Where no scope is configured at all, the surface degrades gracefully to the unfiltered behaviour rather than failing.
Export
Catalog export reads the real catalog chain, schemas to tables to columns, through the audited read paths. Every read is a local query: no network, no cloud and no third-party dependency, so export holds single-cell and multi-cell, sovereign and air-gapped, at the core and at the edge.
The formats, and why the list can be trusted
| Format | What it renders |
|---|---|
| Summary JSON (default) | Catalog identity plus schema and table counts |
| JSON | The structured export: per-schema CREATE TABLE DDL plus the summary |
| DDL | Concatenated CREATE TABLE text for the whole catalog |
| CSV | The flattened column inventory: schema, table, column, type, nullable |
| Parquet | The same column inventory as real Apache Parquet columnar bytes, so a catalog's structure loads straight into a data tool |
| OpenLineage | The catalog's captured lineage as OpenLineage RunEvent documents, round-trippable back through the import path |
The format is selected on the export call, and the advertised list and the rendered list are the same list by construction: the endpoint that enumerates formats and the endpoint that produces them both read one definition, so the two cannot drift apart.
Two behaviours are worth quoting to an evaluator:
- An unknown format is a rejected request, not a silent fallback to the default. Asking for something that does not exist tells you so.
- OpenLineage with no lineage source wired refuses, deny-closed, rather than returning an empty run as though it had succeeded. An empty success is the more dangerous answer, because it looks like "there is no lineage" when it means "nothing was asked."
There is no Apache Atlas export. A faithful projection onto its type system is real design work, and a shallow mapping that produced Atlas-shaped output without honouring that system would be worse than none, because it would import cleanly and mean something different. If Apache Atlas is a requirement, treat it as not present and plan on the lineage formats above instead.
Related
- The catalog and lineage, the catalog this surface exposes.
- Search and discovery, the native discovery surfaces.
- In-engine enforcement, the policy plane the external face re-enters.
- Interoperability, the wider both-directions integration story.