Fine grain reward system evaluates full search workflows effectively
F$^{2}$DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows
Computation and Language
Summary
Many AI systems solve complex questions by searching and thinking step-by-step. Existing ways to judge their answers look only at the final response, not the whole process. The authors propose F2DR, a method that scores every step in this search and answer process to give a better overall evaluation. They also created a new benchmark dataset to test reward models on these multi-step tasks. Their tests show their method works better than current ways at judging AI workflows throughout the entire process.
What this means in practice
- •For enterprise ai teams: Improve quality control by evaluating AI query-solving workflows at every step for better end results in customer support and data analysis.
- •For ai model developers: Develop and test reward models that better assess iterative AI reasoning and retrieval workflows to boost model performance.
Authors
Bojian Xiong, Wentao Ding, Yujing Lu, Shaowei Zhang, Ling Shi, Jing Liao, Yan Wang, Yueyang Zhang, Long Xia, Zhiyuan Sun, Daiting Shi, Jingzhou He, Yuqi Ren, Deyi Xiong
Abstract
With the widespread industrial deployment of Large Language Models (LLMs), DeepSearch has emerged as the dominant paradigm for resolving complex user queries. It typically operates through an iterative closed-loop workflow consisting of planning and reflection, information retrieval, and answer generation. However, existing reward models (RMs) and evaluation benchmarks are primarily designed for static single-turn tasks, failing to capture the full-pipeline complexity of DeepSearch workflows. To address this limitation, we propose F2DR, a fine-grained full-pipeline DeepSearch reward framework. F2DR evaluates DeepSearch workflows across three dimensions: Content, Trajectory, and Answer, enabling comprehensive process-level assessment. We further construct DeepSearch RM-Bench, a dedicated benchmark for evaluating RMs in DeepSearch scenarios. Extensive experiments demonstrate that F2DR achieves significantly higher evaluation consistency than self-evaluation-based baselines, while DeepSearch RM-Bench exhibits strong discriminative capability across existing open-source RMs. We will publicly release the complete DeepSearch RM-Bench dataset soon.