Search state awareness improves large language model searches by 24 percent
Self-Assessment Reinforcement Learning for Search-State Awareness in Retrieval-Augmented Large Language Models
Computation and LanguageArtificial Intelligence
Summary
Large language models can use internet searches to find information and answer complex questions, but they sometimes don't know what they still need to find, causing mistakes or stopping too soon. The authors created a new method called Self-Assessment Reinforcement Learning (SARL) that helps the model understand what information it has and what it still needs during searches. SARL guides the model to search more effectively and efficiently without extra checks while running. This approach improved performance by about 24% compared to strong past methods and remains stable even as tasks get more complicated.
What this means in practice
- •For software developers: Improve intelligent assistants that retrieve and combine external information to answer complex, multi-step queries more accurately and thoroughly.
- •For knowledge management teams: Enhance internal search tools to better track what information has been found and what remains needed in research workflows.
Authors
Taewhoo Lee, Minju Song, Jungwoo Park, Chanwoong Yoon, Junha Jung, Hyeon Hwang, Sanghoon Lee, Minbyul Jeong, Jaewoo Kang
Abstract
Recent advances in search agents have enabled large language models to tackle complex tasks by retrieving and synthesizing external information. However, existing search agents often lack search-state awareness, i.e., the ability to recognize what information has been obtained and what remains missing, leading to hallucinated reasoning and prematurely terminated search. To address this limitation, we propose Self-Assessment Reinforcement Learning (SARL), a reinforcement learning framework in which models learn to recognize their own search state during training and improve their search behavior accordingly, without requiring additional assessment at test time. SARL achieves this through a dual reward design that captures the corrective effect of each assessment while penalizing excessive reliance on it, jointly encouraging effective and efficient search trajectories. Under this setup, SARL achieves a 24.3% relative improvement over the strongest baseline, with average gains of 35.9% on multi-hop reasoning and 27.9% on deep research benchmarks against the strongest per-benchmark baselines. SARL further surpasses methods built on auxiliary supervision or multi-agent coordination, despite operating as a standalone RL framework with no inference-time scaffolding. Beyond final performance, SARL maintains training stability under extended search horizons, a regime where existing methods consistently degrade, enabling sustained gains as task complexity grows.