Multi perspective reasoning improves text ranking accuracy and robustness
Think Thrice Before Reranking: Multi-perspective Evidence and Reasoning Integration for Text Reranking
Information RetrievalComputation and Language
Summary
Current methods that use large language models to rank documents usually follow a single line of reasoning, which can lead to mistakes and miss different important clues. The authors created MERIT-Rank, a system that looks at multiple reasoning paths to better decide which documents are most relevant to a query. They also developed a training method that gradually improves these reasoning paths and ranking quality. Tests show MERIT-Rank outperforms several strong competitors, even using smaller language models.
What this means in practice
- •For search engine developers: Improve the accuracy and robustness of search result rankings by integrating multiple reasoning perspectives into document evaluation.
- •For digital library managers: Enhance the quality of document retrieval in digital libraries by applying multi-trajectory reasoning for better relevance assessment.
Authors
Lijun Liu, Zhengzong Chen, Wenyan Li, Yuanyuan Zhao, Fei Huang
Abstract
Reasoning-based reranking with Large Language Models (LLMs) has shown promising improvements in text ranking. However, current methods predominantly rely on a single reasoning trajectory, resulting in rankings that are susceptible to reasoning errors and inherently constrained in modeling the multifaceted signals underlying document relevance. To resolve this dilemma, we propose MERIT-Rank(Multi-perspective Evidence and Reasoning Integration for Text Reranking), a framework that models complementary reasoning trajectories to improve reranking robustness. MERIT-Rank formulates a Multi-Trajectory Reasoning Space (MTRS) that evaluates query-document relevance from multiple perspectives and introduces a joint reranker that consolidates these reasoning paths into a unified ranking decision. We further develop Progressive Rank Policy Optimization (PRPO), a progressive training framework that stabilizes reasoning trajectories while continually improving ranking quality through staged optimization objectives. Experiments on both reasoning-intensive and traditional retrieval benchmarks show that MERIT-Rank consistently achieves superior performance over competitive baselines. The 4B model notably outperforms most 7B and even 32B rerankers on BRIGHT.