Agent aware reranker improves search quality and efficiency
ACARE: Agent-Context-Aware Reranker for Iterative Search
Information Retrieval
Summary
When computer programs search for information, they often break a question into smaller parts to find answers. But sometimes the parts they search don’t fully help answer the original question. The authors created ACARE, a tool that looks at the whole question and the search process, not just the small parts, to better rank useful information. This makes searches more accurate and faster, especially for smaller AI programs. Their work shows how understanding the bigger picture can lead to better search results.
What this means in practice
- •For search system developers: Integrate ACARE to improve document ranking using both agent queries and overall task context, enhancing search accuracy and speed.
- •For chatbot developers: Use ACARE to boost smaller conversational agents’ ability to find precise and relevant information, narrowing the gap with larger models.
Authors
Sahel Sharifymoghaddam, Yauhen Babakhin, Reza Esfandiarpoor, Oliver Holworthy, Jimmy Lin, Bo Liu
Abstract
In agentic search, a document can match an agent-generated sub-query without helping answer the user's original question. Conventional rerankers score documents against the sub-query alone, overlooking the broader task and the agent's reasoning. We introduce the Agent-Context-Aware Reranker (ACARe), a compact 4B LLM trained to rerank documents using this context when available. Across multiple benchmarks and agent families, ACARe improves retrieval quality over its base reranker while reducing search effort. These benefits extend to answer generation: ACARe improves accuracy and reduces searches over no reranking, while matching or exceeding the base reranker's mean accuracy with more precise evidence. It also helps smaller agents narrow the retrieval-quality gap to larger agents, showing how better retrieval tools can make smaller agents more effective. The model, training data, and code will be released soon.