Unified Data Foundation → Metadata Layer

Data Dictionary

Human-readable definitions for every table and field — what it means, not just what type it is.

High-Level Design

The Data Dictionary adds meaning on top of the technical schema.

Data Source
Data Sources
Every touchpoint and business system
→
Ingestion
Ingestion Layer
SDKs, connectors, protocols
→
Processing
Event Schema & Registry
Technical schema; dictionary adds business meaning
→
Foundation
Data Dictionary
Business-readable field definitions in Azure Purview
→
Intelligence
Semantic Layer
Query & Analytics Engine surfaces definitions to business users
→
Activation
Self-Service Analytics
Business users understand fields without asking engineering

💼 Business Context

  • A schema tells you a field's type; a dictionary tells you what it means and how to use it correctly — both are necessary for self-service analytics
  • Reduces the "what does this column actually mean" questions that otherwise land on data engineering
  • Owned by Analytics Engineering, with definitions contributed by each business domain

🔌 Technical Overview

The data dictionary layers business-readable descriptions, valid value lists, and usage notes on top of the technical schema stored in the Catalog, managed through Azure Purview's business glossary feature. Fields can be linked to a canonical business term (e.g., 'Customer Lifetime Value') with a single definition shared across every table that includes that concept, rather than each table's schema comment drifting independently.

Dictionary Contents

Business-readable descriptions Valid value lists Canonical term linking Usage notes/caveats

💾 Dictionary Entry

{
  "field": "marts.customer_ltv.ltv_band",
  "business_term": "Customer Lifetime Value Band",
  "description": "Tier bucket derived from trailing-12-month LTV",
  "valid_values": ["bronze", "silver", "gold", "platinum"]
}

🔗 Integration Points

  • Azure Purview business glossary — the dictionary implementation
  • Catalog — technical schema that the dictionary adds meaning on top of
  • Query & Analytics Engine's semantic layer — surfaces definitions in BI tools
  • Domain teams — contribute and own definitions for their business terms

🧰 Services Consumed

  • Owning microservice — Cxos.Foundation.Api (see the Full Application Service Map)
  • Database — ADLS Gen2 (Iceberg) + Azure Database for PostgreSQL (policy/retention state)

⚠️ Non-Functional Considerations

  • Scale: dictionary entries grow with business term count, not data volume — a curation effort, not an infrastructure concern
  • Latency: not applicable — this is reference metadata, not a runtime dependency
  • Reliability: canonical term linking prevents definition drift across tables that reuse the same concept
  • Security/Privacy: dictionary entries can flag a field as PII-adjacent even before formal classification is applied

🎯 Enterprise Example

A marketing analyst building a self-service dashboard hovers over 'ltv_band' and sees its business definition and valid values directly in the BI tool, instead of pinging data engineering to ask what the field means.

← Back to Metadata Layer