GraphRAG when vector search is not enough:
relational questions need structure

Why similarity search stalls on relational questions

Dense retrieval excels when a good answer lives in one or two passages that look like the question. Enterprise questions often do not. "Who owns service X in region Y after the reorg?" is an edge walk across people, teams, and systems.

Vector indexes return paragraphs that mention the service name. They do not guarantee the ownership edge is current, complete, or connected to the right region node. Fluency then fills the gap with a confident owner who is wrong, or with a plausible-sounding chain that never existed in any system of record.

Operators see this in on-call handoffs: the chat bot cites an old runbook paragraph naming last quarter's team, while the CMDB already shows a new owning squad. Similarity ranked the familiar text; it never checked whether the ownership edge was still valid.

GraphRAG is not a fashion upgrade. It is a different retrieval contract for relational jobs. Keep classic hybrid RAG for topical work; see production RAG. Use graphs when the question is about entities and edges, not only topical similarity.

Failure modes of vector-only stacks on graph-shaped work

Invented joins. Two entities appear in neighboring chunks or in the same meeting notes. The model treats co-mention as an edge. A support agent then tells a customer that Team A owns a dependency Team A never owned. No retrieval score flagged the missing relationship because the chunks looked topically relevant.

Stale org truth. People move; chunks still say old owners; no graph update invalidated the edge. After a reorg, FAQ evals may still pass because procedure text did not change. Ownership and escalation questions quietly fail until a human notices the wrong page or Slack handle in the answer.

Partial multi-hop. One hop retrieves well (find the service), the second hop has no structured neighbors to expand (region, owner, on-call). The model invents the second hop from prior training or from unrelated passages. Path length grows in the narrative while grounding shrinks.

Eval mismatch. FAQ and single-passage procedures score strong. Ownership, lineage, and "which systems depend on X" cases are rare in the golden set, so the dashboard stays green while relational jobs fail in production.

ACL leakage via expansion. Multi-hop walks that ignore node-level identity can surface a neighbor document the user cannot open. Vector-only stacks already struggle with ACL; graph expansion without the same filters multiplies the risk. Treat this as a hard fail, same class as agentic RAG ACLs.

The hybrid control model

Extract or maintain entities and relationships with an explicit ontology: services, owners, policies, environments, regions. Prefer curated edges for high-stakes relations (ownership, access, compliance bindings). Use extraction for breadth, then route uncertain edges into a review queue instead of trusting every LLM-extracted link.

Operable steps most teams need:

1. Name the entity types and edge types you will retrieve. Keep the set small enough that operators can audit it.
2. Assign owners for each high-stakes edge type. Ownership of the graph is not the same as ownership of the wiki.
3. Use vectors to find entry nodes or seed passages from the user question.
4. Traverse under hop limits, ACL filters on nodes and edges, and a max subgraph token budget.
5. Pack a compact subgraph (node IDs, edge labels, timestamps) plus cited passages into context. Prefer structured path citations over a wall of prose.
6. Version the graph. When an ownership edge changes, the assistant should see the new edge without waiting for every document rewrite.

Evaluation for relational retrieval

Build cases that require two and three hop ownership, dependency, and policy lineage. Label expected node paths, not only final sentences. Include negative cases where a tempting co-mention must not become an edge.

Score path correctness and edge freshness separately from prose quality. A confident answer with the wrong owner is a hard fail, even if the writing is clear. Track hop-depth distribution so you know whether production questions actually need the graph or are still FAQ-shaped.

Apply the same identity filters on graph nodes that you apply on chunks. Multi-hop graph walks need the same discipline as agentic RAG ACLs. Add regression cases that flip an ownership edge and assert the assistant updates within a defined freshness window.

Separate the eval suites: topical FAQ stays on hybrid vector retrieval; relational suites gate GraphRAG changes. Mixing them hides regressions behind easy wins.

Ontology governance operators can run

Without governance, GraphRAG becomes a second shadow database. Document which systems are sources of truth for each edge type (HR for people, CMDB for services, policy registry for controls). Define how extraction errors get corrected: who reviews, what evidence is required, and how wrong edges are tombstoned rather than left to age out silently.

Keep a changelog for ontology changes. Adding a new edge type is a product decision, not only a modeling convenience. If operators cannot explain why a node exists, the assistant will invent semantics around it.

For metric relationships in analytics, the cousin problem is the generative BI semantic layer: structure and contracts beat freeform joins.

Rollout and measurement

Start with one domain: service ownership or policy lineage. Prove operators trust a path inspector that shows nodes and edges used, with timestamps and ACL scope. Do not roll GraphRAG into every channel until that inspector is usable in support and on-call workflows.

Measure path accuracy, hop-depth, stale-edge incidents, and time to update critical relationships after org changes. Pair those with user-facing trust metrics: rate of "wrong owner" tickets and time to correct an edge after a reorg.

Ship in shadow mode first: log the graph path beside the vector-only answer without changing the user-visible response. Compare path correctness offline. Only then route relational intents to the hybrid packer.

Document ontology owners and how extraction errors get corrected. GraphRAG without that operating model does not fail loudly; it fails as confident, relational fiction. Keep classic hybrid RAG for topical work, and reserve the graph for jobs where the answer is the path.