EviBack: Search-Agent Reinforcement Learning via Evidence-Constrained Teacher Backoff
2026-07-27 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors introduce EviBack, a method that helps reinforcement learning systems improve multi-turn search by giving extra guidance when no helpful signals are present. EviBack keeps the assessment of evidence separate from refining answers, so it doesn’t mistakenly ignore missing evidence. They use an automated process assisted by GPT-5.5 to create a better teaching system that improves answer quality and reduces unnecessary searches. Tests across several question-answering benchmarks show that EviBack leads to more accurate and valid answers.
Reinforcement learningAgentic RAG systemsMulti-turn searchEvidence assessmentTeacher backoffGPT-5.5Open-domain QAF1 scoreSearch-R1Task decomposition
Authors
Xiao Ma, Zhiquan Hu, Yi Wei, Chenchen Zhao, Yijun Chen, Jicheng Zhao, Yuming Li Chuang Dai
Abstract
Reinforcement learning enables Agentic RAG systems to learn multi-turn search from verifiable outcome rewards, but all- zero rollout groups provide no comparative signal and may hide useful search behavior. We present EviBack, an evidence- constrained Teacher backoff that supplies auxiliary super- vision to such groups while preserving verifiable Actor re- wards. It separates evidence assessment from answer refine- ment, preventing reference answers from overriding evidence- insufficiency judgments. A fully automated, end-to-end GPT- 5.5-assisted APE pipeline starts from a manually authored single-prompt dual-task Teacher, automatically partitions and labels rollout data, and performs ablation, task decomposition, evaluation, and selection to produce a gated two-stage Teacher. Compared with the manual design, the resulting Teacher im- proves downstream F1 and valid-answer rate while reduc- ing search, duplicate queries, and forced termination. Across seven open-domain QA benchmarks and three Qwen3 scales, EviBack improves F1 over Search-R1 and raises both single- and multi-hop macro F1. We guarantee that the code will be made publicly available at a later stage.