GraphRAG

What it is / what's different

RAG is a technique that has AI look up and consult relevant content in external documents before answering. Typical RAG retrieves short text chunks similar in meaning to the question, but this approach struggles with questions that require scanning everything, such as 'What are the main themes of this entire collection?', or that require connecting information scattered across many places. GraphRAG does not stop at simply chunking documents; it extracts the entities and relationships within them and organizes them into a web-like knowledge graph. The key difference is that it uses this structure for both retrieval and answering.

How it works

Taking Microsoft's published GraphRAG as the reference, documents are first split into small units suitable for analysis. Next, an LLM extracts entities and relationships from each chunk to build a knowledge graph. Closely connected entities are grouped hierarchically into clusters called 'communities', and a summary is written for each group in advance. When a question comes in, partial answers are generated from each group summary, and these partial answers are then combined into the final answer.

How it has developed

'From Local to Global: A Graph RAG Approach to Query-Focused Summarization', published in 2024 by Darren Edge and colleagues, is the paper that popularized the name GraphRAG; it proposed a new way to answer questions spanning an entire document collection using knowledge graphs and community summaries. RAPTOR, released the same year, proposed repeatedly clustering and summarizing text chunks to build a tree-shaped summary structure, then retrieving summaries at multiple levels. GRAG proposed a method for documents that are inherently interconnected, such as citation networks or social media, that efficiently finds relevant small subgraphs and feeds both the text content and the connection structure to the LLM. All of these studies are part of a trend toward using structured information to address 'the limitation of RAG that only retrieves short chunks'.

Why it's getting attention

Demand is growing for companies to have AI work with data the model never saw during training, such as internal documents, research materials and work records. Such material involves many 'questions that connect multiple pieces of information' and 'questions that summarize the whole', so ordinary RAG alone is sometimes not enough. Microsoft stated that GraphRAG showed substantially better results than conventional RAG on these two types of questions. Graph database companies are also promoting GraphRAG as a knowledge base for AI agents.

Things to know (limitations)

Building the graph and writing summaries in advance requires many LLM calls, which costs money and time up front. If the LLM extracts entities or relationships incorrectly, those errors can remain in the graph and carry through to answers. When documents change, the graph and summaries must be updated again. For simple fact-checking questions, ordinary RAG is often sufficient, so it is better to choose between them or combine them depending on the type of question.

Example

For example, suppose a company has connected several years of meeting minutes and reports to an AI assistant. A question like 'What risk was discussed most often last year?' cannot be answered from a single sentence. With GraphRAG, it can produce an answer spanning all the documents, based on a graph of relationships among projects, departments and risks and on topic-level summaries. Microsoft has released GraphRAG as an open-source tool, and graph database companies such as Neo4j also offer features for building GraphRAG.

Related terms

RAG Knowledge graph Vector DB Embedding Chunking Agentic RAG

References

• From Local to Global: A Graph RAG Approach to Query-Focused Summarization (2024)
• GRAG: Graph Retrieval-Augmented Generation (2024)
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024)
• GraphRAG official documentation (Microsoft)
• What is GraphRAG? (Neo4j)