Unified Data Foundation → Metadata Layer

Policies & Entitlements

The metadata-driven layer that determines who can access which tables and fields — access control expressed as data, not hardcoded application logic.

High-Level Design

Policy as data means it is enforced consistently everywhere.

Data Source
Data Sources
Every touchpoint and business system
→
Ingestion
Ingestion Layer
SDKs, connectors, protocols
→
Processing
Governance & Security
Defines the policy rules enforced here
→
Foundation
Policies & Entitlements
Metadata-driven access rules in Azure Purview
→
Intelligence
Access Control (RBAC/ABAC)
Enforces these policies at query time
→
Activation
Every Query & Consumer
Policy is enforced consistently regardless of access path

💼 Business Context

  • Centralizing access policy as metadata means it's enforced consistently everywhere, instead of each tool implementing its own access checks
  • Makes access audits tractable — policies are inspectable data, not scattered application code
  • Owned by Data Governance / Security

🔌 Technical Overview

Policies are expressed as metadata attached to catalog entries in Azure Purview — which Azure AD groups or roles can read which tables/columns, often keyed off the classification tags set during schema governance. The Query & Analytics Engine and any other lakehouse-reading service checks these entitlements at query time via a shared policy-evaluation library, rather than each service reimplementing its own access logic.

Policy Types

Table-level access Column-level masking Row-level filtering Purpose-based entitlements

💾 Entitlement Policy

{
  "table": "curated.customer_profile",
  "column": "email",
  "policy": "mask",
  "allowed_roles": ["data-engineering", "customer-support-tier2"]
}

🔗 Integration Points

  • Azure Purview — policy/entitlement metadata storage
  • Azure AD (Entra ID) — role and group source of truth
  • Shared policy-evaluation library — used by every lakehouse-reading service
  • Governance & Security's Access Control — the enforcement mechanism this metadata drives

🧰 Services Consumed

  • Owning microservice — Cxos.Foundation.Api (see the Full Application Service Map)
  • Database — ADLS Gen2 (Iceberg) + Azure Database for PostgreSQL (policy/retention state)

⚠️ Non-Functional Considerations

  • Scale: policy evaluation is a metadata lookup, not a per-row computation, keeping overhead low even on large tables
  • Latency: policy checks add negligible latency to query execution
  • Reliability: centralized policy means a single update propagates everywhere immediately
  • Security/Privacy: policy metadata itself is tightly access-controlled to prevent privilege escalation

🎯 Enterprise Example

A new customer-support tool needs read access to customer profiles, but email addresses should stay masked for tier-1 agents. The entitlement policy is defined once and enforced automatically for every service that queries the table, rather than the support tool needing its own bespoke masking logic.

← Back to Metadata Layer