Decisionrank speeds up document ranking with better on-policy training
DecisionRank: On-Policy Distillation for Decision-Model Rerankers
Computation and Language
Summary
Ranking documents quickly and accurately is important for tasks like search engines. The authors show a new way to train a decision-based model, called DecisionRank, that learns directly from a powerful generative model's preferences. Their method creates rankings based on scores, then gets feedback from the teacher model to improve these scores. This approach leads to faster and sometimes more accurate ranking than the original generative model, even when using fewer computing resources.
What this means in practice
- •For search engine engineers: Replace slow generative rankers with faster decision-based models that maintain or improve ranking accuracy in document retrieval tasks.
- •For information retrieval system builders: Train efficient rerankers that directly optimize ranking scores through teacher model feedback to enhance reasoning-intensive search benchmarks.
- •For advertising technology teams: Accelerate candidate item ranking for ads by training decision-model rankers that capture preferences from larger generative models but run much faster.$Commercial implications: Enables faster ad ranking systems maintaining generative model quality, improving real-time bidding and personalization.
Authors
Yanran Wu, Dian Yu, Chongyang Bai, Jieyi Zhang, Rui Liu, Tianyi Chen, Shike Mei, Hang Cui, Frank Shyu, Sandeep Pandey, Luke Simon, Chunwei Liu, Xi Liu
Abstract
Decision models target structured tasks by directly predicting choices or scores, avoiding autoregressive decoding latency. However, transferring a generative LLM's ranking knowledge to a decision model remains challenging: the teacher expresses preferences through a ranked list of document ID sequences, while the student determines the ranking using scores from the first-token position. To bridge this gap, we introduce DecisionRank, a decision model trained through item-space on-policy distillation (OPD). Item-space OPD constructs the student's rollout by sorting its document scores into a ranked sequence. A generative teacher takes the resulting ranking list as inputs and provides conditional feedback to directly supervise those scores. Combined with offline ranking supervision, DecisionRank achieves state-of-the-art performance on both general and reasoning-intensive retrieval benchmarks such as TREC DL, BEIR, and BRIGHT. Across four benchmarks, item-space OPD consistently outperforms alternative training objectives. Even at 1.7B parameters, DecisionRank surpasses its 27B generative teacher on TREC DL19--23. At 8B, it outperforms its generative listwise counterpart while running $24.5\times$ faster on TREC DL and $16.7\times$ faster on BRIGHT.