Lightweight ranker improves multi-viewpoint evidence selection under budget
Learning Marginal Viewpoint Utility for Budgeted Evidence Selection in RAG
Information Retrieval
Summary
When answering questions with many different viewpoints, it's important to gather evidence that covers all perspectives without exceeding a fixed limit on information. The authors developed LMVU, a simple tool that learns to pick which pieces of evidence add the most new viewpoints as more information is gathered. They tested LMVU on several benchmarks and found it outperforms other methods in covering diverse perspectives within tight size limits. This approach helps ensure a balanced set of answers without overwhelming the system's capacity.
What this means in practice
- •For question answering system developers: Improve selection of evidence covering diverse viewpoints efficiently under strict context size limits to enhance answer quality.
- •For knowledge management teams: Support automated document selection that ensures broad perspective coverage while respecting processing constraints in large information repositories.
Authors
Yiyang Duan, Ziqi Yu, Tiantong Wu, Yebo Wu, Wei Yang Bryan Lim
Abstract
Retrieval-augmented generation over multi-viewpoint queries must preserve distinct answers and positions from the retrieved evidence while operating under a fixed context budget. We introduce LMVU, a lightweight ranker that learns to prioritize candidates by the marginal viewpoint coverage they add to the current selection. Training uses viewpoint annotations from each benchmark's development split to compare candidates within the same selection state. At inference, LMVU repeatedly scores budget-feasible candidates as the context grows, without accessing these annotations. We evaluate two forms of recovered viewpoint coverage: RVC$_1$ assigns each viewpoint full credit for at least one supporting source unit, while RVC$_2$ assigns half credit for one and full credit for two or more distinct source units. Exact oracles measure achievable coverage under the same evidence pool and budget. Across four viewpoint-annotated benchmarks at a 1,024-token budget, LMVU achieves gains of up to 0.047 and 0.052 points in RVC$_1$ and RVC$_2$, respectively, over the strongest non-oracle baseline for each benchmark and metric, and reaches the exact viewpoint-breadth ceiling on two benchmarks.