Agent controls long web searches with memory and verification states
Traverse: Learning When to Remember, Reset, and Redirect for Long-Horizon Web Search
Computation and Language
Summary
Sometimes when searching for lots of information on the web, computers can make mistakes early on and then keep those mistakes, making it hard to find the right answers later. The authors created a computer agent that organizes its search into three steps: setting rules for good answers, finding answers, and then checking those answers before stopping or looking more. They also gave the agent a memory tool to keep track of useful information while searching. Training the agent was tricky because it could forget how to use its memory well, but the authors solved this by changing how they taught the agent to do the final steps after managing memory.
What this means in practice
- •For web search engineers: Build search systems that autonomously check and refine answers during multi-step web queries to reduce early mistakes and improve final results accuracy.
- •For financial investigation teams: Use agents that manage memory and verify findings to improve accuracy and reliability in complex multi-step online information searches.
Authors
Jingyuan Ma, Lynx Aster, He Zhang, Siyao Song, Weijie Yuan, Zhe Zhang, Kai Jia, Zhifang Sui
Abstract
Long-horizon information-seeking agents often accumulate noisy or misleading context, causing early mistakes to persist and making recovery increasingly difficult. We introduce an autonomous search harness in which the agent manages its own search process through three states: Rubric, Answer, and Verify. The agent first defines criteria for a valid answer, searches under these criteria, and then independently verifies the result before deciding whether to terminate or continue searching. It is further equipped with a Seal Memory tool that enables active context management. Training this behavior with reinforcement learning, however, can induce Seal Collapse, resulting in unstable training and preventing the agent from reliably learning when and how to use its memory tools. We solve this with a simple strategy that trains only the final segment after context management. Our 35B model achieves 72.83 on BrowseComp, outperforming comparable open-source systems, and consistently improves over the base model across BrowseComp-ZH, xbench, DeepSearchQA, WideSearch, financial investigation, and product search. Ablations show that autonomous compression outperforms automatic compaction and validate our RL design.