Key takeaways
- Put agents on the semantic layer, not raw warehouses.
- Metric catalogs are the source of truth for definitions.
- Verify SQL and tool plans before charts reach decision-makers.
- Explainability is part of the product for finance and operations.
What a generative BI intelligence layer is
A generative BI intelligence layer is the combination of a governed semantic layer, a metric catalog, agent planners, verification, and delivery surfaces. People ask data questions in natural language. The system answers only with certified metrics, filters, and grains that already exist.
It is not a chatbot with a warehouse password. If an agent can invent a join, a filter, or a KPI name that no owner published, you do not have an intelligence layer. You have fluent SQL with no contract.
Knackline treats this as a production system the inheriting data team can operate: definitions have owners, rejected plans are visible, and a wrong number can be reconstructed from the metric path.
The KPI hallucination problem
Natural language feels productive until two teams get two different revenue numbers from the same question. The model did not invent malice. It invented a join or a filter that looked plausible.
Dashboards already encode hard-won definitions. Generative BI that bypasses those definitions recreates shadow metrics at chat speed. Finance cannot tell whether "bookings" meant signed contract, invoice, or cash.
The job is not "chat with the database." The job is "ask questions that respect the company's metric contracts." Operators should be able to open the metric path and see the same definition the weekly ops dashboard uses.
What inventing a metric looks like in production
A typical miss is a question that sounds simple: "What was revenue last quarter in EMEA?" An unconstrained agent may:
- Sum a fact table that mixes bookings and billings
- Filter on a region column that is not the commercial region used in the catalog
- Change the time grain from fiscal quarter to calendar quarter
- Drop a currency conversion the semantic layer already owns
Each of those looks like a successful query. Latency is fine. The chart renders. The inheriting team still cannot defend the number. That is the failure demos hide.
Agents on a governed semantic layer
Expose metrics, dimensions, and entities through a semantic layer the agent must use. Prefer certified metrics over freeform table access. If the catalog does not contain a metric, the agent should refuse or ask for a definition, not invent one.
Encode ownership and freshness. If a metric is deprecated or delayed, the agent should say so instead of improvising a substitute. A stale certified metric is still more honest than a fresh invented one.
Access control travels with the layer. Row-level and column-level rules must apply to agent-generated queries the same way they apply to BI tools. If an analyst cannot see a customer segment in Looker, the agent must not retrieve it through SQL.
Loading diagram…
Verification loops before narratives ship
Generate a plan: which metrics, which filters, which time grain. Validate the plan against the catalog before executing SQL. Treat a rejected plan as a success when the catalog cannot support the question.
Show the query or metric path next to the chart. Analysts and finance partners need to audit the path, not only the picture. If they cannot reconstruct the number from the catalog, the answer is not shippable.
Allow draft mode. Agents can propose analyses; humans publish when the stake is high. Weekly ops reviews, board packs, and compensation metrics stay behind a review gate until definition adherence is proven on a failure set.
What an intelligence layer actually contains
A metric catalog, semantic models, agent planners, verification, and delivery surfaces (chat, briefings, alerts). Without the catalog, you only have a chatbot with a warehouse password.
Observability belongs here too: which questions failed, which metrics were missing, which plans were rejected. That backlog is your next modeling work. The inheriting team should own L1 (who watches rejected plans), L2 (who fixes a broken metric), and L3 (who changes a definition).
Netrasight-style AI-assisted analytics is an early form of this idea: ask questions of data without standing up an entire parallel BI stack first, while still aiming at governed answers.
A rollout sequence that reduces risk
Start with a narrow domain: one team, one certified metric set, read-only answers. Expand after evals show stable definition adherence. Do not open write-back, alerting, or board-facing briefs until the failure set stays green across a release.
Measure definition adherence, not only answer satisfaction. A liked wrong number is worse than a refused question. Score whether the agent selected a certified metric, whether the grain matched, and whether the metric path was shown.
Keep a human review queue for financial and board-facing narratives until trust is earned with evidence. The inheriting team should be able to halt a surface without waiting on the vendor who shipped the demo.
FAQ
- What is a generative BI intelligence layer?
- It is the combination of a governed semantic layer, metric catalog, agent planners, verification, and delivery surfaces that lets people ask data questions without inventing KPIs or bypassing access controls.
- How do you stop NL-to-SQL agents from inventing metrics?
- Require agents to select certified metrics from a catalog, validate plans before execution, and show the metric path alongside the answer so humans can audit definitions.
- Is generative BI a replacement for dashboards?
- Not usually. It accelerates routine questions and exploration on top of the same governed definitions dashboards already use. Dashboards remain useful for shared, recurring views.