BoundaryMORPH boosts document selection within limited reranking budgets
BoundaryMORPH: Budgeted Reranking via Active Set Selection for Diffuse Retrieval
Information Retrieval
Summary
Modern systems that answer open-ended questions often need to gather many documents, but the computer can only carefully check a small number of those documents. The authors found that existing methods spend too much effort reviewing the easiest top choices and miss useful documents further down the list. They created BoundaryMORPH, an algorithm that smartly focuses expensive checks on documents near the cutoff point to fit the best set in a limited space. By using a model that learns from past checks, BoundaryMORPH spreads information efficiently and improves the overall selection quality.
What this means in practice
- •For search engine developers: Improve document ranking in systems with limited compute budgets for verifying retrieved results to better fit answers in restricted context windows.
- •For conversational ai builders: Enhance retrieval quality for open-ended queries by efficiently selecting diverse relevant documents within strict latency constraints.
Authors
Eylon Caplan, Shamik Roy, Shib Sankar Dasgupta, Yingfan Wang, Rashmi Gangadharaiah
Abstract
Open-ended queries in modern Retrieval-Augmented Generation (RAG) are increasingly "diffuse," requiring a large set of documents to be assembled into a finite LLM context window. To ensure retrieval quality, systems use fast dual-encoders and more expensive cross-encoders (CEs) to score candidates. However, the CE budget $B$ is strictly bounded by latency and is often smaller than the context window capacity $k$. This mismatch makes standard reranking structurally flawed: it wastes compute verifying obvious top candidates while ignoring relevant documents further down the initial ranking. To address this, we introduce BoundaryMORPH, a novel algorithm that allocates CE budget specifically for the LLM's context capacity $k$. Using a Gaussian Process, BoundaryMORPH treats the initial dual-encoder ranking as a structural prior and intelligently spends CE calls on resolving top-$k$ set membership at the boundary, rather than seeking a single most-relevant document. Information from each CE call propagates to unscored documents, maximizing the utility of the budget. We demonstrate that BoundaryMORPH achieves state-of-the-art set retrieval quality across multiple models and datasets with open-ended queries ($+5.4$ nCG@100 over the strongest baseline).