Re ranker improves candidate scoring with reasoning based queries
Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval Heads
Computation and Language
Summary
Ranking answer choices usually relies on just the question, which limits how well relevant information can be found. The authors propose a new system called Re$^3$Rank that first creates a step-by-step reasoning trace combining the question and answer options. Then it uses this reasoning to better focus on important parts when scoring each candidate. This joint approach helps the system pick better answers and outperforms many existing methods across several benchmarks.
What this means in practice
- •For search engine developers: Improve ranking of search results by integrating reasoning steps into the ranking process.
- •For virtual assistant teams: Enhance answer selection by conditioning retrieval on generated reasoning traces rather than fixed queries.
Authors
Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo
Abstract
Attention-based rerankers score each candidate by the attention mass that retrieval heads route to it from the question, but the question, serving as a fixed query, carries limited information for locating relevant evidence. We propose Re$^3$Rank, a listwise reranker built on reasoning-conditioned retrieval heads (RRHeads). A generator first writes a reasoning trace over the question and the candidates, and a scorer then uses the trace tokens as the attention query of a few selected RRHeads, reading relevance as continuous scores. We present a straight-through estimator to propagate the ranking loss through the discrete trace, so that the generator and the scorer can be trained jointly for ranking. Averaged over 9 benchmarks, Re$^3$Rank outperforms 11 baseline rerankers: our 4B variant exceeds the strongest baseline by 3.11 NDCG@10 points on average, and even our 0.6B variant surpasses rerankers with up to 32B parameters.