Graph structure cuts cost and improves answers in document-based question systems
When Is Graph Structure Worth Its Cost? The Case for Structure Pricing in Retrieval-Augmented Generation
Artificial Intelligence
Summary
Answering questions that need info from many documents can be slow and costly if the system reads everything fully. The authors made EffiRAG, a method that uses a map (graph) of document parts to quickly find important text to answer questions. This saves money and keeps answers better by not losing the original info. Their tests show EffiRAG gives better answers more often than a similar system while spending less than half the money on language model calls.
What this means in practice
- •For enterprise search teams: Build efficient question-answering systems that combine multiple documents with fewer costly AI calls to reduce operating expenses.
- •For digital library developers: Deploy graph-based methods to quickly locate relevant info within large collections while preserving original document context.
Authors
Yuzhong Zhang, Haoyang Ma, Chao Peng, Lionel Briand, Boxi Yu, Jialun Cao
Abstract
Graph-based retrieval-augmented generation (RAG) can help answer questions that require information from many documents. However, building a graph often requires many language-model calls during ingestion. It is therefore important to ask whether its quality gains justify the additional cost. We present EffiRAG, a graph-based RAG system designed to reduce this cost. It uses the graph to locate relevant passages and generates answers from the original text. This design preserves source information while keeping graph construction and query processing lightweight. We evaluate EffiRAG on UltraDomain, which contains 120 open-ended questions from four domains. Compared with LightRAG-hybrid, EffiRAG produces the preferred answer on 93 questions. LightRAG is preferred on 7, and the remaining 20 are splits. EffiRAG also reduces total system cost by 57 percent, from USD 0.952 to USD 0.408. The cost includes language-model calls during ingestion and querying. The advantage remains as the corpus grows. At 10 and 20 documents per domain, EffiRAG uses a lightweight, non-LLM filter to skip low-salience chunks. It remains preferred over LightRAG-hybrid. It costs 4.2 times and 4.5 times less, respectively. The comparisons identify different quality-cost trade-offs. Graph-based RAG systems should therefore be evaluated by both answer quality and cost. The results favor graph structure that locates and preserves source evidence.