Coverage aware retrieval improves long form question answering results
Decompose, Estimate, Select: Coverage-Aware Context Construction for Long-Form RAG
Information Retrieval
Summary
Answering complicated questions that need information from many parts can be tricky because search results might repeat the same facts and miss some important details. The authors came up with a method that breaks big questions into smaller ones, then carefully picks diverse documents that cover different parts of the question without much repetition. They also taught the system to check if it fully used the found information, and then to improve the answer by using leftover facts it missed at first. This method made answers more complete and relevant without needing extra labeled data.
What this means in practice
- •For search engine developers: Create question answering systems that return more comprehensive and less redundant evidence by selecting documents with subquestion coverage.
- •For knowledge base maintainers: Improve automated responses by generating more complete answers using the model’s self-check to incorporate overlooked evidence.
Authors
He Tu, Aleena Bosco Kurumthottam, Torsten Suel
Abstract
Retrieval-augmented generation (RAG) grounds a language model's output in retrieved evidence, but for complex questions spanning multiple aspects of an information need, relevance-based retrieval may return redundant evidence while leaving some aspects unsupported. We propose a coverage-aware retrieval pipeline that decomposes queries into subquestions, ranks candidates with a coverage-trained bi-encoder, predicts a document\,$\times$\,subquestion coverage matrix with a cross-encoder, and selects documents by greedy maximum-marginal coverage. On TREC RAG 2024, our approach improves $α$-nDCG@10, which rewards newly covered nuggets and discounts redundant evidence, from $0.625$ to $0.702$, approaching the $0.718$ gold-nugget upper bound without sacrificing relevance. We then find that ground-truth oracle retrieval, despite substantially higher retrieval coverage, delivers no more nugget coverage to the answer than our estimated selection. This suggests that answer completeness depends not only on what evidence is retrieved, but also on how effectively the generator uses it. To address this, we introduce a \emph{grounded self-feedback loop} in which the model identifies underused evidence from its first answer and generates again without gold supervision, raising $V_{\mathrm{strict}}$, the fraction of vital reference nuggets fully supported by the answer, by $0.086$ over two passes while largely preserving faithfulness.