Enterprise GraphRAG: Implementation Patterns for Production

What can't vector search alone do?

Cosine similarity between a question and a stored chunk is the wrong operation for a surprising share of the questions enterprise users actually ask. It cannot count or aggregate: a question whose answer is a number should come from a query, not a retrieval, or the model will confidently report an artifact of how many chunks it happened to be shown. It cannot retrieve absence: “which accounts have no assigned owner” has no embedding to match against. It has no opinion about recency or supersession, so an older, more detailed record can outscore the current one. And it cannot enforce permissions without leaking, because filtering after retrieval is inherently unsafe.

None of that means vector search is wrong: it's excellent at finding a relevant entry point. The failure is asking it to also assemble the structured context around that entry point, which is a job it was never built to do.

What is the anchor and expand pattern?

Anchor and expand splits retrieval into two phases with two different jobs. The anchor phase uses vector or full text search, scoped from the start to what the requesting user can access, to surface a handful of candidate entry points. Because a meaningful share of the raw top k can be inaccessible to any given user, over fetching before the permission filter runs (and monitoring how many candidates survive it) is what keeps answer quality from silently degrading.

The expand phase then does the exact part: typed traversal outward from each anchor, sibling records in the same context, the entity it evidences, who approved it, and any later record that superseded it. The result is serialized with an identifier attached to every fact, so an answer generation prompt can require the model to cite the id behind every claim: and a downstream check can regex the output and verify every cited id was actually present in the context. That turns “hallucination testing” from a vibe into something that runs in CI.

Should you let an LLM write graph queries in production?

No: not against production data without hard controls, and the reasons aren't hypothetical. A model composing its own query can write something syntactically valid that bypasses every permission scoping rule in your retrieval layer. It can issue an unbounded traversal that never returns on a large, densely connected graph. Any record in your corpus is untrusted input the moment an account can write to it, which makes prompt injection a real attack surface, not a theoretical one. And a model can silently reverse the direction of a relationship in a query that runs fine, returns rows, and gives an answer that is exactly backwards: the failure mode that survives testing precisely because nothing errors.

The fix isn't a better prompt. It's removing the capability: expose a small set of parameterised, reviewed query tools. The model picks which question to ask and supplies parameters: it never composes a graph pattern itself. Every query is in version control and has been through code review, which means “what can this assistant actually do” is answerable by reading a file, not by trusting a prompt.

How do you evaluate a GraphRAG system properly?

A single relevance score is close to meaningless for this class of system. The failure modes that matter (multi hop reasoning, aggregation, absence, supersession, permissions, and provenance) need to be tested as named classes with a known correct answer each, against a small, deliberately constructed fixture graph.

The permission class deserves the most attention of all: run the entire evaluation suite twice, once as a fully authorized user and once as a user outside the relevant boundary, and diff the two runs. Any fact that appears in both runs and shouldn't (including a count, a title, or the mere confirmation that something exists) is a leak, and it should gate CI. That diff is the test that protects the project from the one incident that actually ends it.

What does a GraphRAG query actually cost?

Graph traversal against a well indexed graph is fast: typically single digit to low double digit milliseconds for the anchor and expand phases combined, which is a rounding error against end to end latency. The answer generation call that follows dominates both latency and cost by a wide margin. The practical consequence is architectural: engineering time spent shaving milliseconds off a query that already runs quickly returns very little. Time spent retrieving less but more precise context: a smaller model for a sufficiency check, caching anchors keyed on both the question and the viewer's permission scope (never on question text alone, which is a cross tenant leak waiting to happen): moves the number that actually matters.