Semantic layer contracts for generative BI agents

Why a metric list is not a contract

The generative BI intelligence layer report argues agents must sit on a governed semantic layer. Teams often stop at a wiki of metric names. Agents still improvise joins because nothing machine-checks the plan.

A contract is enforceable. It states metric IDs, grains, allowed dimensions, owners, freshness, and deprecation. The agent either satisfies the contract or refuses. Documentation without enforcement is how shadow KPIs return through chat, faster than any dashboard ever allowed.

Operators see the gap in weekly reviews: two narratives, two revenue numbers, both fluent, both citing "the warehouse." Neither cites a certified metric ID. The catalog existed. The agent was never required to use it.

The job is shared truth at conversation speed, not a smarter way to bypass the warehouse governance you already fought for. Contracts make that job machine-checkable.

Failure modes when contracts are soft

Silent substitutes. Deprecated revenue_v2 is swapped for an unofficial sum the model likes. The chart looks fine. Finance cannot reconcile it to the certified definition. Nobody notices until board prep.

Grain lies. Daily metrics get compared as monthly without disclosure. The narrative says "up 12%" while the grains are incomparable. Soft contracts treat grain as a prompt suggestion instead of a type error.

Lineage theater. The UI shows a SQL string nobody maps to a certified metric ID. Auditors get text, not a contract path. The appearance of explainability replaces actual adherence.

Access theater. The agent queries with a service role that bypasses row-level rules analysts live under. The answer is numerically "correct" for a privilege the user does not have. That is a security incident dressed as analytics.

Deprecation drift. Old metric IDs remain callable with no end date and no disclosure. Agents prefer familiar IDs. The catalog says "prefer new," but the runtime never enforces the preference.

The contract model agents must satisfy

Version metrics like APIs. Breaking changes get new IDs. Deprecations announce end dates. Agents prefer current certified IDs and disclose when forced onto a legacy metric during transition.

Validate plans before execution: metric IDs, filters, grain, and time range must type-check against the semantic model. Reject freeform SQL for high-stakes domains. Allowlisted analytical patterns can exist, but they still must bind to metric IDs the contract knows.

Attach lineage to every answer: metric IDs, owner, as-of freshness, and access scope. If lineage cannot be shown, do not show the number. A chart without lineage is a demo artifact, not a decision artifact.

Operable steps:

1. Publish metric IDs with grain, dimensions, owner, freshness SLA, and deprecation status.
2. Require the agent plan to cite those IDs before any warehouse execution.
3. Run a contract validator that fails closed on unknown measures, illegal grains, or blocked dimensions.
4. Execute only through the governed path; strip or refuse freeform SQL in certified domains.
5. Render lineage beside the chart or narrative, including as-of time and access scope.

Refusal, evals, and change management

Refusal is a product feature. "No certified metric for that" beats a confident wrong board number. Offer closest certified metrics when helpful, without inventing a new one. Clarifying questions are allowed; silent improvisation is not.

Eval definition adherence in CI: plans must cite valid IDs, and answers must not introduce unofficial measures (eval gates). Score refusal correctness as carefully as answer satisfaction. A system that never refuses will look helpful while leaking shadow KPIs.

Observe which questions hit refusal. That backlog is your next modeling work, visible through agent observability. Treat repeated refusals as a product queue for the metric owners, not as evidence that the agent should "try harder" with freeform SQL.

Change management belongs in the contract: who can add a metric, who can deprecate, how agents learn new IDs, and how long legacy IDs remain callable with disclosure. Without those rules, the catalog and the runtime drift apart again.

Rollout and metrics

Pilot one domain with strict contracts and read-only answers. Pick a domain where finance already argues about definitions; that pressure makes adherence measurable. Expand after adherence scores stay high and finance partners can audit lineage without a separate investigation.

Shadow mode helps: log which metric IDs the agent would have used, compare against analyst-approved paths, then enforce. Do not flip enforcement on every domain at once. A soft contract in one domain will undermine trust in the hard contract next door.

Measure definition adherence rate, refusal rate, time-to-add a certified metric, and incidents of unofficial KPI use in agent answers. Pair those with freshness breach counts when answers ship past SLA. Review refusal clusters in the same meeting that reviews new metric requests so modeling work stays tied to real questions.

Document who owns metric versions and how agents learn deprecations. Semantic layer contracts turn generative BI from a clever query writer into a governed interface on the same truth dashboards already use. The catalog stops being documentation. It becomes the runtime boundary.