Search and discovery
A catalog nobody can search is an inventory, not a platform. This page is how an analyst, an engineer or an agent finds what exists, and why what they find is scoped to what they are allowed to know about.
The surfaces
| Surface | What it does |
|---|---|
| Full-text search | Over catalog assets, kept current by a background indexer rather than rebuilt on demand |
| Faceted search | Narrow by domain, owner, classification, asset kind, quality grade and tags |
| Hybrid search | Lexical and semantic together, so "the table with customer churn" finds something even when no column is named churn |
| Autocomplete | As you type |
| A command palette | Keyboard-first navigation across the catalog |
| Federated discovery | Across cells, so a search answers from the estate rather than from one cell |
The asset search index is maintained by background workers that reindex and run catalog anti-entropy, which is why a newly registered asset becomes findable without anyone running a crawl.
Ranking knows more than keywords
Because the catalog already scores its assets, discovery can use that rather than guessing.
- Quality scoring across seven weighted dimensions (completeness, freshness, documentation, governance, usage, quality, lineage), graded A+ through F.
- Usage analytics, so a heavily used, well-documented asset outranks an abandoned one with a similar name.
- Collaborative-filtering recommendations, so people working on similar things surface each other's assets.
- Auto-enrichment with personal-data detection across nine pattern families and domain inference across nine domains, so an asset arrives with useful facets before anyone curates it.
That last one matters more than it sounds. The reason most catalogs are hard to search is that nobody fills in the metadata, so enrichment that runs on ingest is the difference between a catalog that gets used and one that gets abandoned.
Discovery respects classification
Search is an exfiltration path if you let it be. Two controls apply.
Results are scoped to the caller. Search runs under the same authorization as a read, so an asset the caller may not know about does not appear.
Lineage visibility is viewer-aware. A catalog-authoritative permission check decides what of the lineage graph a given viewer may see, so the provenance graph does not become a side channel that reveals the existence of data you cannot read.
The second one is the subtle one. A lineage edge pointing at an asset you cannot see still tells you it exists, who owns it, and roughly what it feeds. Filtering the graph rather than only the assets is what closes that.
Breaking changes find you
Discovery works in the other direction too. Breaking-change detection and a propagation notifier tell downstream owners when an upstream change affects them, and impact analysis answers "what breaks if I change this" before the change rather than after.
Together with column-level lineage extracted by parsing queries, that turns the catalog from a place you look things up into a system that tells you when something you depend on is about to move.
Related
- The catalog and lineage, what is being searched.
- The semantic layer, metric definitions that give search something meaningful to match.
- Data products, the curated end of discovery.
- Discovery & mapping, pointing the same machinery at an estate the platform does not yet run.