Understanding the Behaviors of Environment-aware Information Retrieval
2026-06-15 • Computation and Language
Computation and LanguageInformation Retrieval
AI summaryⓘ
The authors studied how language models (LLMs) can learn to ask better questions depending on which tool (retriever) they use to find information. They found that different retrieval tools work best with different question styles, like descriptive or question-like queries. Using reinforcement learning, the model can adapt its questions to fit each retriever, improving performance. They also introduced a new training method to help the model learn more stably over multiple steps. This work helps make systems that combine language models with search tools more effective.
Retrieval-augmented generation (RAG)Large Language Models (LLMs)Reinforcement Learning (RL)Query FormulationInformation RetrievalRetriever CharacteristicsTraining StabilityBranching RolloutRetriever-aware SystemsMulti-step Trajectories
Authors
Ruifeng Yuan, Chaohao Yuan, David Dai, Yu Rong, Hong Cheng, Hou Pong Chan, Chenghao Xiao
Abstract
Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.