Deep search agents improved for longer complex queries and reasoning
LexiHorizon: Stabilizing Reinforcement Learning for Long-Horizon Deep Search
Artificial Intelligence
Summary
Finding information by asking many questions can get tricky when the search tool sometimes misses important details due to slight changes in wording. The authors created a method called LexiHorizon that helps the search process work better over long sequences of questions by keeping track of what was found recently and rewarding careful searching. Their approach was tested on several benchmarks, showing clear improvements in finding correct answers compared to earlier methods. This means search agents can better handle complicated tasks requiring many steps of thinking and looking up information.
What this means in practice
- •For knowledge engineers: Build advanced search agents that maintain reasoning steps and handle extended querying over diverse data sources.
- •For software developers: Develop tools that improve complex information retrieval by managing long sequences of queries and responses more effectively.
Authors
Zhiqing Nong, Liang Wen, Chao-Hsuan Liu
Abstract
Deep search agents tackle complex knowledge tasks through iterative retrieval, multi-hop reasoning, and evidence synthesis across multiple sources. Existing approaches typically assume relatively stable retrieval systems and operate over short-horizon tool interaction. However, when retrieval is sensitive to query formulation, even a semantically appropriate query may fail to surface critical evidence because of mismatched entity names, aliases, or keyword combinations. Recovering from such failures requires repeated query reformulation and longer interaction trajectories. This setting poses a distinct training challenge, as the policy must sustain long-horizon query exploration while managing an expanding volume of retrieved content. We propose LexiHorizon, a framework for training search agents over long horizons that expands the trajectory context budget, manages accumulated retrieval content using a window over recent tool observations while preserving the reasoning history, and introduces an outcome-gated search-effort reward that provides a bounded bonus for tool invocations to trajectories with nonzero answer reward. Experiments on XBench, WebWalkerQA, and BrowseComp-ZH show that the resulting 9B model consistently outperforms both its base model and MiroThinker-1.7-mini, with maximum absolute gains of 8.7 and 23.8 percentage points, respectively. These results suggest that combining an extended context budget with reasoning-preserving context management benefits long-horizon deep search agents.