GraphRAG Architecture Guide

Architecture Series · Part 4 of 4

TL;DR

• GraphRAG Architecture extends Enterprise RAG by building a knowledge graph from document corpora — entities, relationships, and community summaries — then retrieving through graph structure instead of vector similarity alone.

• Local search answers entity-specific questions; global search answers dataset-wide themes — the query planner selects the mode. Using only one mode is the most common production failure.

• Community detection and graph summaries are index-time investments — Leiden clustering and LLM-generated community reports enable global queries that flat chunk retrieval cannot answer.

• Graph indexing is a separate lifecycle from query serving — incremental updates, versioned graph snapshots, and community regeneration run on different schedules than the online retrieval path.

• GraphRAG complements — not replaces — Enterprise Knowledge Graph Architecture — GraphRAG extracts graphs from documents; EKG governs enterprise source-system facts. Production systems often use both.

Why This Matters

GraphRAG Architecture is the production system design for relationship-aware retrieval over document corpora. If Enterprise RAG retrieves chunks and Knowledge Graph + LLM Architecture grounds reasoning in governed enterprise graphs, this guide covers the graph extracted from documents — indexing pipelines, community detection, local/global search, and operational controls at scale.

Who should read this?

Architecture

System overview

GraphRAG Architecture is the system design for deploying graph-based retrieval-augmented generation in production. If you have read Enterprise RAG Architecture, you understand hybrid retrieval, gateways, and operational controls. If you have read Knowledge Graph + LLM Architecture and Enterprise Knowledge Graph Architecture, you understand governed graph platforms and LLM grounding patterns. This document answers: how do you engineer a GraphRAG system that survives enterprise load and answers questions flat RAG cannot?

GraphRAG is not a replacement for the prior three guides. It is the relationship-aware retrieval layer for document corpora where:

Assume readers understand GraphRAG fundamentals. This guide covers production architecture — indexing pipelines, query planning, storage, operations, and when not to deploy.

Reference Architecture

Source: Microsoft GraphRAG

A production GraphRAG system is a composed architecture of two major paths:

Source: Microsoft GraphRAG

Production teams treat graph indexing as infrastructure with its own SLA — not a nightly script that blocks query serving when it fails.

Diagram: GraphRAG production architecture

flowchart TB
    subgraph index [Indexing offline]
        DOC[Documents] --> CH[Chunk + extract]
        CH --> GS[Graph store]
        CH --> CD[Community detection]
        CD --> REP[Community reports]
        CH --> VEC[Vector index]
    end
    subgraph query [Query online]
        Q[User query] --> PL[Query planner]
        PL --> LS[Local search]
        PL --> GL[Global search]
        PL --> HY[Hybrid retrieval]
        LS --> GS
        GL --> REP
        HY --> VEC
        LS --> RANK[Evidence ranking]
        GL --> RANK
        HY --> RANK
        RANK --> LLM[LLM + guardrails]
    end

Indexing and query services scale independently; community reports power global search.

Diagram: Community detection hierarchy

flowchart BT
    L0[Level 0 communities — broad themes] --> L1[Level 1 — sub-themes]
    L1 --> L2[Level 2 — entity clusters]
    L2 --> E[Entities + relationships]

Leiden hierarchical clustering enables global search at the appropriate abstraction level.

Architecture principles

How We Got Here

Architecture maturity progression

GraphRAG is the capstone retrieval pattern in the DataAIHub Architecture Series Version 1.

Enterprise Search
    ↓
Traditional RAG
    ↓
Enterprise RAG
    ↓
Knowledge Graph + LLM
    ↓
GraphRAG
    ↓
Agentic GraphRAG

Enterprise Search indexes siloed content. Traditional RAG retrieves chunks. Enterprise RAG adds security, hybrid retrieval, and operations. Knowledge Graph + LLM grounds reasoning in governed enterprise graphs. GraphRAG (this guide) extracts and queries graph structure from document corpora for relationship-aware retrieval. Agentic GraphRAG adds multi-step retrieval planning, tool use, and iterative graph exploration — the next evolution beyond fixed local/global search.

Diagram: Enterprise retrieval maturity

timeline
    title GraphRAG in the architecture series
    section Search
        Enterprise Search : Keyword indexes
        Traditional RAG : Vector chunks
    section Platform
        Enterprise RAG : Security + hybrid + ops
        KG + LLM : Governed graph grounding
    section Graph
        GraphRAG : Communities + multi-hop
        Agentic GraphRAG : Iterative graph exploration

GraphRAG is the capstone retrieval pattern when document corpora contain rich relationships.

Deploy GraphRAG only after Enterprise RAG fundamentals work. GraphRAG indexing costs more and latency is higher — justify with query types flat RAG cannot answer.

The Problem GraphRAG Architecture Solves

Teams that deploy GraphRAG without understanding why encounter failures that Enterprise RAG troubleshooting does not address.

Important

GraphRAG failures are indexing and planning failures — stale graphs, wrong search mode, poor extraction quality — not prompt problems.

Poor multi-hop retrieval

Vector search returns chunks similar to the query. Multi-hop questions require entities connected across documents — "How does Policy A relate to Incident B through Vendor C?" No single chunk contains the path. GraphRAG traverses extracted relationships.

Disconnected chunks

Flat RAG assembles top-k chunks that may contradict or duplicate without structural context. GraphRAG retrieves a connected subgraph — entities and relationships that form a coherent evidence package.

Missing relationships

If entity extraction misses edges during indexing, retrieval cannot recover them at query time. Graph construction quality is the ceiling on GraphRAG answer quality.

Entity ambiguity

"Apple" as company vs. fruit, or "Jordan" as person vs. region — unresolved mentions retrieve wrong subgraphs. Entity linking on the query path is mandatory.

Context fragmentation

Stuffing many chunks into LLM context wastes tokens on irrelevant text. GraphRAG compresses evidence through community summaries and ranked subgraphs — higher signal per token.

Long context windows

Larger context does not fix structural blindness. A 128K window full of chunks still lacks explicit relationship topology. Graph structure is the compression mechanism.

Hallucinations

When graph retrieval returns insufficient evidence, LLMs fabricate connections. Production architecture enforces refusal behavior and faithfulness checks against retrieved graph evidence.

Poor explainability

Regulated and high-stakes use cases need evidence beyond "chunk 7." GraphRAG provides entity IDs, relationship types, community report citations, and source document provenance.

Indexing Pipeline

Documents
 │
Chunking
 │
Entity + Relation Extraction
 │
Graph Construction
 │
Community Detection
 │
Graph Summaries
 │
Graph Index

Comparisons

GraphRAG Architecture extends — not replaces — Enterprise RAG and complements Knowledge Graph + LLM Architecture for governed facts. Use these tables to choose retrieval patterns and justify index cost.

Traditional RAG vs GraphRAG

Traditional RAG

User
 │
Vector Search
 │
Top-k Chunks
 │
LLM
 │
Answer

GraphRAG

User
 │
Query Planner
 │
Local / Global Search
 │
Graph Evidence
 │
Evidence Ranking
 │
LLM
 │
Grounded Answer

Adopt GraphRAG when evaluation shows flat RAG fails on global and multi-hop question classes — not because graph technology is trending.

GraphRAG vs governed Knowledge Graph + LLM

GraphRAG indexing vs fine-tuning

Decision tree: GraphRAG deployment

Decision tree: When to deploy GraphRAG Architecture

flowchart TD
    A[Enterprise RAG in production?] -->|No| B[Stabilize RAG first]
    A -->|Yes| C[Eval fails global or multi-hop?]
    C -->|No| D[Stay on Enterprise RAG]
    C -->|Yes| E[Budget for 5-20x index cost?]
    E -->|No| F[Agentic RAG or query expansion]
    E -->|Yes| G[Deploy GraphRAG Architecture]
    G --> H[Hybrid local + global + vector]

Deploy GraphRAG when eval proves chunk-only retrieval fails — not because graphs are trending.

Component breakdown

Engineering Insight

Engineering Tip: Separate indexing and query services before choosing a framework. The indexing pipeline will run 10–100x longer than any single query — different resource profiles, different failure modes.

Document Processing

Purpose: Ingest, normalize, and prepare source documents for chunking and extraction.

Responsibilities:

• Load PDF, HTML, DOCX, markdown, and API-sourced content
• Extract clean text; preserve metadata (title, date, author, ACL)
• Deduplicate and version documents
• Apply document-level access classification

Technology choices: Unstructured.io, Apache Tika, custom parsers, LlamaIndex readers, Azure Document Intelligence.

Production considerations: Preserve source URI and document ID through entire pipeline. OCR quality affects extraction — monitor error rates per document type.

Common mistakes: Stripping metadata needed for ACL filtering. Processing duplicate documents without dedup keys.

Chunking

Purpose: Segment documents into text units sized for entity extraction and embedding.

Responsibilities:

• Split documents with overlap for context continuity
• Create TextUnit records linked to parent documents
• Tune chunk size for extraction model context limits
• Log chunk-to-document provenance

Technology choices: LangChain splitters, LlamaIndex node parsers, Microsoft GraphRAG chunk workflow.

Production considerations: GraphRAG extraction quality is sensitive to chunk boundaries — entities split across chunks may get duplicate or incomplete nodes. See Chunking Strategies.

Common mistakes: Chunks too large for extraction model. No overlap — relationships spanning boundaries missed.

Entity Extraction

Purpose: Identify named entities and concepts from text units using LLM or NER models.

Responsibilities:

• Extract entities with types (Person, Organization, Concept, etc.)
• Assign stable entity IDs within indexing run
• Merge duplicate entity mentions within document
• Output entity table for graph construction

Technology choices: Microsoft GraphRAG extract_graph, GPT-5.6 / Claude Sonnet 5 extraction prompts, spaCy + LLM hybrid, Neo4j GraphRAG LLM entity extraction.

Production considerations: Cache LLM extraction calls (Microsoft GraphRAG cache layer). Sample and audit extraction quality per corpus. Version extraction prompts.

Common mistakes: No human audit of extraction on new corpora. Changing extraction prompts without re-indexing.

Relationship Extraction

Purpose: Identify typed relationships between extracted entities.

Responsibilities:

• Extract edges with relationship types and descriptions
• Capture relationship strength or confidence if supported
• Link relationships to source text units for provenance
• Deduplicate parallel edges

Technology choices: Microsoft GraphRAG relationship extraction, custom LLM prompts, Neo4j GraphRAG, RDF triple extraction pipelines.

Production considerations: Relationship types should be constrained by schema or post-validated — uncontrolled types produce noisy graphs.

Common mistakes: Accepting every LLM-extracted edge without validation. No provenance link to source chunk.

Graph Construction

Purpose: Assemble entities and relationships into a queryable knowledge graph structure.

Responsibilities:

• Build graph tables (nodes, edges) per GraphRAG knowledge model
• Merge entities across text units and documents
• Apply entity resolution across document boundaries
• Version graph snapshots

Technology choices: Microsoft GraphRAG parquet outputs, Neo4j, NetworkX (prototype), Memgraph, Stardog (materialized graph).

Production considerations: Entity merge strategy is critical — aggressive merge collapses distinct entities; weak merge fragments the graph. Document as ADR.

Common mistakes: Rebuilding entire graph daily without incremental merge. No graph version tag on queries.

Knowledge Graph

Purpose: Persist the extracted graph for traversal, community detection, and query-time retrieval.

Responsibilities:

• Store nodes, edges, communities, and reports
• Support fast neighbor lookup and subgraph retrieval
• Index entity embeddings for similarity search
• Replicate for HA

Technology choices: Neo4j, Memgraph, Stardog, Microsoft GraphRAG parquet + vector store, Azure Cosmos DB (Gremlin).

Production considerations: Choose storage aligned with query patterns — Neo4j for Cypher traversal, parquet + LanceDB for Microsoft GraphRAG default stack.

Common mistakes: Graph storage not sized for edge count growth. No backup of graph snapshots.

Community Detection

Purpose: Cluster densely connected entities into hierarchical communities for global search.

Responsibilities:

• Run Leiden (or similar) hierarchical clustering on graph
• Produce community hierarchy at multiple levels
• Assign entities to community memberships
• Regenerate on graph updates above threshold

Technology choices: Microsoft GraphRAG detect_communities (graspologic), NetworkX + custom, Neo4j GDS community algorithms.

Production considerations: Community detection is compute-intensive — run offline. Tune resolution parameter for community granularity (ADR). See Figure 1.

Common mistakes: Communities too granular (noise) or too coarse (useless summaries). No regeneration policy when graph grows.

Graph Summaries

Purpose: Generate LLM-written community reports that summarize themes, entities, and relationships per community.

Responsibilities:

• Produce hierarchical community reports at each level
• Include key entities, relationship patterns, and findings
• Embed report text for retrieval
• Version reports with community structure

Technology choices: Microsoft GraphRAG generate_reports, custom LLM summarization per community subgraph.

Production considerations: Summary quality drives global search — evaluate with human reviewers on sample communities. Cache report generation.

Common mistakes: Stale reports after graph update. Reports generated from incomplete community membership.

Graph Index

Purpose: Unified retrieval index over entities, relationships, community reports, and text units.

Responsibilities:

• Embed and index all retrievable artifacts
• Support vector search over entities and reports
• Maintain mapping from index entries to graph elements
• Incremental index updates on graph changes

Technology choices: LanceDB (Microsoft GraphRAG default), Pinecone, Weaviate, Elasticsearch, Neo4j vector indexes.

Production considerations: Pin embedding model version — re-embed on model change. Separate indexes for local (entity) vs. global (report) retrieval if needed.

Common mistakes: Index drift from graph store. Single embedding model for entities and long reports without testing.

Local Search

Purpose: Answer entity-specific questions by retrieving ego networks around relevant entities and linked text units.

Responsibilities:

• Identify seed entities from query (entity linking)
• Expand to k-hop neighborhood or ranked subgraph
• Include linked text unit chunks for detail
• Return structured local context package

Technology choices: Microsoft GraphRAG local search, Neo4j GraphRAG vector + traversal, custom Cypher expansion.

Production considerations: Cap hop depth and node count — supernodes degrade latency. Default for "who," "what," entity-centric questions.

Common mistakes: Unbounded traversal. Local search for global thematic questions (wrong mode).

Global Search

Purpose: Answer dataset-wide thematic questions using community reports at appropriate hierarchy level.

Responsibilities:

• Map query to relevant community level
• Retrieve top community reports by embedding similarity
• Optionally drill down hierarchy for detail
• Assemble global context for LLM

Technology choices: Microsoft GraphRAG global search, custom community report retriever.

Production considerations: Global search depends on community report quality — invest in indexing evaluation. Higher latency than local — set expectations.

Common mistakes: Using global search without community reports. Wrong hierarchy level selected.

Hybrid Search

Purpose: Combine local entity retrieval, global community retrieval, and vector chunk search in one query path.

Responsibilities:

• Run local and global paths in parallel where appropriate
• Merge evidence with reciprocal rank fusion or learned ranker
• Deduplicate overlapping entities and chunks
• Route by query planner decision

Technology choices: Microsoft GraphRAG mixed mode, LangChain ensemble retriever, custom orchestration.

Production considerations: Hybrid is default for ambiguous queries. Log which paths contributed evidence for debugging.

Common mistakes: Always running both paths (cost). Never running global (misses thematic questions).

Query Planner

Purpose: Classify query intent and select local, global, or hybrid retrieval strategy.

Responsibilities:

• Analyze query type (entity-specific vs. thematic vs. multi-hop)
• Select search mode and parameters (hop depth, community level)
• Decompose complex queries into sub-queries if needed
• Log planning decisions for evaluation

Technology choices: Economy-tier LLM classifier (Claude Haiku, Gemini 3.7 Flash), rule-based patterns, Microsoft GraphRAG search type parameter.

Production considerations: Planner errors cause wrong search mode — maintain golden set of query → mode pairs. Allow user override for power users.

Common mistakes: No planner — always local search. Planner latency without timeout.

Graph Context Builder

Purpose: Assemble retrieved graph evidence into structured context for the LLM within token budget.

Responsibilities:

• Serialize subgraph as structured JSON or natural language
• Include community report excerpts when relevant
• Attach provenance (document IDs, entity IDs, community IDs)
• Truncate by relevance rank within token cap

Technology choices: Custom templates, Microsoft GraphRAG context builders, LangChain document formatters.

Production considerations: Structured JSON evidence enables faithfulness checking. Cap tokens — ranked evidence beats volume.

Common mistakes: Dumping raw graph tables into prompt. No token budget enforcement.

Prompt Builder

Purpose: Construct generation prompt with system instructions, graph context, and citation requirements.

Responsibilities:

• Load versioned prompt templates
• Inject graph context and query
• Require citation format for entities and reports
• Support streaming generation config

Technology choices: Langfuse prompt management, Git-backed templates, Microsoft GraphRAG prompt configs.

Production considerations: Log prompt_version per trace. Separate prompts for local vs. global answer styles.

Common mistakes: Same prompt for all search modes. No instruction to refuse when evidence empty.

LLM

Purpose: Generate natural language answers grounded in assembled graph evidence.

Responsibilities:

• Synthesize answer from graph context only
• Cite entities, communities, and source documents
• Refuse when evidence insufficient
• Stream tokens for perceived latency

Technology choices: Claude Sonnet 5, GPT-5.6, Gemini 3.5 Pro — routed per workload via AI Gateway; pin model versions per deployment.

Production considerations: Use capable model for synthesis; economy model for planning and extraction. Route through AI Gateway patterns.

Common mistakes: Generation without graph context (degenerates to standard RAG). No refusal behavior.

Evidence Ranking

Purpose: Score and filter retrieved graph evidence before context assembly.

Responsibilities:

• Rank entities, edges, reports, and chunks by relevance
• Deduplicate near-duplicate evidence
• Apply minimum relevance threshold
• Log ranked evidence for evaluation

Technology choices: Cross-encoder reranker, Cohere Rerank, embedding similarity scores, custom graph centrality weights.

Production considerations: Re-ranking graph evidence is high leverage — see Re-ranking. Separate ranking models for entities vs. reports.

Common mistakes: No ranking — passing full neighborhood to LLM. Ranking chunks but not community reports.

Guardrails

Purpose: Validate outputs against retrieved graph evidence and policy rules.

Responsibilities:

• Faithfulness check: claims map to retrieved entities/edges
• Block fabricated relationships not in evidence
• PII and policy filtering on output
• Escalation on low-confidence answers

Technology choices: NeMo Guardrails, custom claim-evidence matcher, LLM-as-judge.

Production considerations: See Guardrails. Verify relationship claims against edge list in evidence package.

Common mistakes: Guardrails only on input. No relationship-level faithfulness check.

Evaluation

Purpose: Measure indexing quality, retrieval mode accuracy, and answer faithfulness continuously.

Responsibilities:

• Golden sets for local, global, and multi-hop questions
• Measure entity extraction precision/recall on sample
• Track search mode classification accuracy
• Run nightly end-to-end eval in CI

Technology choices: RAGAS, DeepEval, Langfuse datasets, custom pytest + LLM judge.

Production considerations: Eval indexing and querying independently. Include questions flat RAG fails — that's GraphRAG's justification.

Common mistakes: End-to-end only. No baseline comparison against Enterprise RAG.

Observability

Purpose: Trace indexing runs, query plans, retrieval paths, and generation costs.

Responsibilities:

• Distributed tracing with trace_id
• Log search mode, seed entities, community IDs retrieved
• Indexing job metrics: extraction counts, community sizes, duration
• Per-tenant cost attribution

Technology choices: Langfuse, OpenTelemetry, MLflow.

Production considerations: Alert on indexing failures, graph staleness, and retrieval empty-rate spikes. See Observability.

Common mistakes: Logging LLM only — not graph retrieval spans. No indexing dashboard.

Query Flow

User
 │
Query Planner
 │
Local / Global Search
 │
Evidence Ranking
 │
LLM
 │
Guardrails
 │
Response

Step-by-Step Flow

End-to-end request flow

User Question
    ↓
API Gateway              → TLS, rate limit, trace_id
    ↓
Authentication           → Token validation, tenant context
    ↓
Intent Analysis          → Classify query type; extract entity mentions
    ↓
Query Planner            → Select local, global, or hybrid; set parameters
    ↓
Entity Linking           → Map mentions to graph entity IDs
    ↓
Local Search             → k-hop subgraph + linked text units (if applicable)
    ↓
Global Search            → Community report retrieval (if applicable)
    ↓
Community Retrieval      → Rank reports; optional hierarchy drill-down
    ↓
Evidence Ranking         → Score and filter entities, edges, reports, chunks
    ↓
Graph Context Builder    → Assemble structured evidence within token budget
    ↓
Prompt Builder           → Load template version; inject context
    ↓
LLM Generation           → Grounded answer with citations
    ↓
Guardrails               → Faithfulness check against evidence
    ↓
Grounded Response        → Answer + graph evidence + trace_id
    ↓
Async: Evaluation        → Sample into eval dataset
    ↓
Async: Observability     → Metrics, cost, latency per stage

Example: "What are the main compliance themes across our vendor contracts, and which vendors are most associated with data residency risks?"

1. Query Planner — Classifies as hybrid: global (themes) + local (vendor entities).
2. Global Search — Retrieves level-1 community reports matching "compliance" and "data residency."
3. Entity Linking — Links "vendors" to Organization entities in graph.
4. Local Search — Expands 2-hop from vendor entities with data_residency_risk edges.
5. Evidence Ranking — Top 3 community reports + top 5 vendor subgraphs.
6. LLM — Synthesizes themes with [community:12] and [entity:vendor_442] citations.

Diagram: GraphRAG query execution sequence

sequenceDiagram
    participant U as User
    participant G as API gateway
    participant P as Query planner
    participant GR as Graph retrieval
    participant V as Vector index
    participant L as LLM
    U->>G: question + tenant
    G->>P: classify local/global/hybrid
    P->>GR: entity link + traverse OR community rank
    P->>V: hybrid chunk search
    GR-->>P: subgraph + reports
    V-->>P: text units
    P->>L: ranked evidence package
    L-->>U: grounded answer + graph citations

Log planner mode and seed entities on every trace — wrong mode is the fastest diagnosis path.

Production Tip

Production Advice: Log search mode and seed entities on every trace. Wrong mode selection is the fastest diagnosis path for bad GraphRAG answers.

Technology Choices

GraphRAG Frameworks

Graph Storage

Embeddings and LLMs

Observability

Architecture decision records

Engineering Insight

Engineering Tip: Document ADR-009 before procurement. Teams buy GraphRAG when they need EKG, or rebuild EKG facts with LLM extraction — both fail.

Production Layers

Documents
 │
Index Pipeline (offline)
 │
Graph Store + Vector Index
 │
Query Service (online)
 │
LLM + Guardrails
 │
Observability

Design Decisions

Production design decisions

Incremental graph construction

Process new and changed documents through extraction and merge into existing graph. Full rebuild only on extraction model version change or catastrophic corruption. Track graph_version on every query.

Graph freshness

Define SLA: community regeneration weekly, incremental document index daily, or event-driven on corpus update. Alert when index lag exceeds SLA. Stale communities produce wrong global answers.

Community regeneration

Trigger when entity/edge count grows > N% or on schedule. Partial regeneration at affected community branches if supported; else full community pass offline.

Graph versioning

Tag snapshots graph-2026-07-11-v3. Query service pins to active version; blue-green switch after eval passes on new version.

Entity resolution

Merge duplicate entities across documents during indexing. Query-time linking maps user mentions to canonical entity IDs. Steward review queue for low-confidence merges.

Cache strategy

Cache community reports (immutable per graph version). Cache local subgraph expansions for frequent entities. Semantic cache for repeated queries per Enterprise RAG patterns.

Evaluation

Golden set with local, global, and multi-hop questions. Compare against Enterprise RAG baseline — GraphRAG must win on target question classes to justify cost.

High availability

Query service: 2+ replicas. Graph store: replication per vendor docs. Indexing: idempotent jobs that can retry without corrupting graph.

Disaster recovery

Security

Document-level ACLs propagated to text units and entities. Filter retrieval by tenant and document access. Audit graph queries in regulated environments.

Tip

Performance Tip: Run local and global search in parallel for hybrid mode. Sequential execution doubles retrieval latency.

Running in Production

Best practices

1. Justify GraphRAG with eval — prove flat RAG fails on your question classes first.
2. Separate indexing from query services — independent scaling and failure domains.
3. Invest in extraction quality — graph ceiling equals extraction ceiling.
4. Use query planner — never default all queries to local search.
5. Version graph snapshots — pin queries to known graph version.
6. Regenerate communities on schedule — stale reports break global search.
7. Entity resolution at index and query time — merge on ingest; link on query.
8. Rank evidence before generation — subgraphs can be large.
9. Structured evidence packages — enable faithfulness guardrails.
10. Cache community reports — immutable per graph version.
11. Incremental indexing — avoid daily full rebuilds at scale.
12. Pin embedding and extraction model versions — document re-index procedure.
13. Hybrid with Enterprise RAG — chunks for nuance, graph for structure.
14. Log search mode and seed entities — essential for debugging.
15. Refusal when evidence empty — never fabricate relationships.
16. Evaluate indexing and querying separately — isolate failure domains.
17. Monitor indexing job health — extraction failures are silent graph rot.
18. Cap traversal depth — prevent supernode explosions.
19. AI Gateway for all LLM calls — extraction, summarization, generation.
20. Compare cost to Enterprise RAG — GraphRAG index cost is 5–20x higher.
21. Use governed EKG for enterprise facts — GraphRAG for document-derived structure.
22. Prompt version every template — log with trace.
23. Red-team relationship hallucinations — faithfulness checks on edges.
24. Document ADRs — especially GraphRAG vs. EKG boundary.
25. Read Parts 1–3 of this series — GraphRAG extends those architectures; it does not replace them.

Best Practice

Best Practice: Run the Production Readiness Checklist before promoting GraphRAG to production traffic.

Common Mistakes

Real Production Example

Example enterprise stack

Indexing runs as scheduled Kubernetes jobs; query service scales horizontally. Governed enterprise facts for operational entities come from Enterprise Knowledge Graph Architecture; GraphRAG indexes the document corpus layer.

Production readiness checklist

• Eval justification — GraphRAG beats Enterprise RAG on target question classes
• Indexing pipeline — Idempotent, monitored, versioned graph output
• Extraction quality — Sampled human audit on entity and relationship precision
• Community detection — Tuned granularity; reports generated and embedded
• Graph versioning — Active version pinned; blue-green deploy tested
• Query planner — Local/global/hybrid routing with logged decisions
• Entity linking — Query mentions mapped to graph IDs with confidence thresholds
• Evidence ranking — Relevance threshold before LLM context assembly
• Guardrails — Faithfulness check on graph evidence
• Incremental indexing — New documents merged without full daily rebuild
• Community regeneration — Scheduled or threshold-triggered
• ACL propagation — Document permissions enforced in retrieval
• AI Gateway — All LLM calls routed with failover
• Observability — Traces for index jobs and query path
• Evaluation pipeline — Nightly golden-set eval in CI
• Disaster recovery — Graph snapshot restore tested
• Cost monitoring — Index and query cost per corpus and tenant

Where It Breaks Down

GraphRAG Architecture fails when teams treat it as "RAG with extra steps" without investing in extraction quality, community regeneration, and query planning. Extraction errors are permanent until re-index — they are not fixable at prompt time.

See GraphRAG for concept-level failure modes. This architecture guide addresses operational failures — versioning, SLA, and eval gates.

When NOT to Use GraphRAG

GraphRAG adds significant indexing cost and query latency. Do not build this architecture when:

• Simple FAQ or policy lookup — Enterprise RAG answers single-document questions faster and cheaper
• Small corpus — under ~500 documents rarely justifies community detection overhead
• No relationships in questions — pure keyword or semantic lookup needs no graph extraction
• Low latency requirements — sub-2s p95 is difficult with graph planning and retrieval
• Prototype — validate retrieval with Enterprise RAG first; prove graph value with eval
• Limited compute budget — indexing requires many LLM calls for extraction and community reports
• Governed enterprise facts only — use Knowledge Graph + LLM over Enterprise Knowledge Graph instead

Use GraphRAG when global thematic questions, multi-hop relationship queries, and corpus exploration are core requirements — and evaluation proves flat RAG fails.

Related Guides

Architecture Series (Version 1):

• Enterprise RAG Architecture — document retrieval platform
• Knowledge Graph + LLM Architecture — LLM grounding in governed graphs
• Enterprise Knowledge Graph Architecture — semantic platform for enterprise systems

Graph foundations:

• GraphRAG — core concepts and indexing overview

• Knowledge Graphs — graph modeling fundamentals

• Graph Databases — storage and operations

• RDF · SPARQL · Cypher

• Cross-cluster retrieval: RAG · Hybrid Search · Agentic RAG · Knowledge Graphs

• Tools: LangChain · LlamaIndex · Neo4j vector

• Rankings: Best Vector Databases · Best AI Agent Frameworks

Diagram: Architecture series learning path

flowchart LR
    A[Enterprise RAG] --> B[KG + LLM Arch]
    B --> C[GraphRAG Arch]
    C --> D[Agentic GraphRAG]

Operations:

• Chunking Strategies — index-time segmentation
• Embedding Models — model selection and versioning
• Re-ranking — evidence ranking patterns
• Evaluation · Observability · Guardrails

Learning Path

Prerequisites: Enterprise RAG Architecture · Knowledge Graph + LLM Architecture · Enterprise Knowledge Graph Architecture · GraphRAG

This guide completes Version 1 of the DataAIHub Architecture Series.

Estimated time: 40 min · Difficulty: Advanced

Architecture Series

Production architecture references for designing and operating enterprise AI systems at scale.

Which Architecture Should You Choose?

Need relationship-aware retrieval?
    ↓
   No → Enterprise RAG
    ↓
   Yes
    ↓
Document corpus or governed enterprise data?
    ↓
Governed enterprise systems → Knowledge Graph + LLM
    ↓
Document corpus
    ↓
Need global themes + multi-hop?
    ↓
   No → Hybrid Enterprise RAG may suffice
    ↓
   Yes
    ↓
GraphRAG Architecture

Need relationship-aware retrieval? — If chunk similarity answers all questions, stop at Enterprise RAG.

Document corpus vs. governed enterprise data? — Source-system facts belong in Enterprise Knowledge Graph Architecture consumed by Knowledge Graph + LLM. GraphRAG targets unstructured document corpora.

Need global themes and multi-hop? — If questions are entity-local only, lighter graph retrieval may suffice. GraphRAG's differentiator is community-level global search and cross-document relationship traversal.

Interview Questions

1. How does GraphRAG Architecture differ from Enterprise RAG Architecture?

   • Expected: builds graph from documents; retrieves entities/communities not just chunks; local vs global search modes.

2. Why separate indexing and query services?

   • Expected: different resource profiles; indexing 10–100× longer; independent failure domains and SLAs.

3. What does community detection enable that vector RAG cannot?

   • Expected: global thematic questions over entire corpus via pre-computed community reports.

4. GraphRAG Architecture vs Enterprise Knowledge Graph Architecture?

   • Expected: GraphRAG extracts from documents; EKG governs source-system facts — complementary (ADR-009).

5. What ADRs matter most on day one?

   • Expected: local/global routing, graph refresh strategy, GraphRAG vs EKG boundary, extraction model versioning.

6. How do you evaluate GraphRAG Architecture against Enterprise RAG?

   • Expected: shared golden set; must win on global/multi-hop classes to justify 5–20× index cost.

7. Why is evidence ranking required before LLM context assembly?

   • Expected: subgraphs and report sets exceed token budget; unranked evidence adds noise and cost.

8. Name three indexing pipeline monitoring signals.

   • Expected: entities per chunk, relationship density, community size distribution, extraction failure rate, graph version lag.

FAQs

How is GraphRAG Architecture different from the GraphRAG fundamentals guide?

The GraphRAG guide explains concepts — indexing phases, local vs. global search, community detection. This guide is the production architecture reference: components, operations, ADRs, checklist, and stack decisions.

How does GraphRAG relate to Enterprise RAG?

Enterprise RAG retrieves document chunks with hybrid search and operational controls. GraphRAG builds a graph from documents and retrieves relationships and communities. Many production systems combine both in hybrid evidence packages.

How does GraphRAG relate to Enterprise Knowledge Graph?

Enterprise Knowledge Graph Architecture governs facts from enterprise source systems (ERP, CRM, MDM). GraphRAG extracts graphs from documents. They are complementary — not substitutes.

Microsoft GraphRAG or Neo4j GraphRAG?

Microsoft GraphRAG implements the research pattern (Leiden communities, local/global search) with parquet outputs — ideal for Azure-aligned teams. Neo4j GraphRAG integrates with production Cypher graphs and GDS — ideal for Neo4j platform shops. ADR-005 documents the choice.

What indexing cost should I expect?

GraphRAG indexing requires LLM calls per chunk (extraction), per community (reports), and embedding passes — typically 5–20x the cost of embedding-only RAG indexing. Budget before procurement.

How often should I regenerate communities?

When entity/edge count grows >10–20% or on a fixed schedule (weekly/monthly). Monitor global search quality — degrading thematic answers signal stale communities.

What latency targets are realistic?

Streaming first token improves perceived latency.

Can I incrementally update the graph?

Yes — merge new document extractions without full rebuild. Community regeneration may be partial or full depending on graph change scope. Full rebuild required on extraction model version change.

How do I evaluate GraphRAG vs. Enterprise RAG?

Shared golden question set. Measure answer correctness, faithfulness, and latency. GraphRAG must win on global and multi-hop classes to justify deployment.

What comes after GraphRAG?

Agentic GraphRAG — iterative retrieval planning, tool use, and multi-step graph exploration. Fixed local/global search is Version 1; agents are the next maturity level.

References

• Microsoft GraphRAG — Indexing Architecture
• Microsoft GraphRAG — Default Dataflow
• Microsoft GraphRAG — GitHub
• Microsoft Research — GraphRAG Blog
• From Local to Global: A Graph RAG Approach (Edge et al., 2024)
• Neo4j — GraphRAG for Agentic AI
• Neo4j GraphRAG Python Library
• LlamaIndex — Knowledge Graph RAG
• LangChain — Graph RAG

Further Reading

• Stardog — Knowledge Graph + LLM Applications
• Memgraph — GraphRAG Patterns
• Microsoft — GraphRAG Solution Accelerator (Azure)

Key Takeaways

• GraphRAG Architecture is the relationship-aware retrieval layer for document corpora — communities, entities, and edges replace chunk-only retrieval for global and multi-hop questions.
• Indexing and query are independent systems — graph construction, community detection, and report generation are offline investments that enable online local and global search.
• Do not retrieve documents; retrieve relationships — the engineering rule that distinguishes GraphRAG from Enterprise RAG.
• GraphRAG complements Enterprise Knowledge Graph Architecture and Knowledge Graph + LLM Architecture — it does not replace governed enterprise semantic platforms.
• This guide completes Version 1 of the DataAIHub Architecture Series — the natural evolution when enterprise knowledge contains rich relationships and multi-hop reasoning becomes essential.