Self-Evolving Deep Research via Joint Generation and Evaluation
2026-06-03 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors created a new way for large language models (LLMs) to improve at writing deep research reports by training the model to both solve and evaluate these reports together. Traditional methods used separate judging systems that couldn’t adapt as the model got better, limiting progress. Their approach, called SCORE, allows the evaluator and generator to learn from each other within the same model, guided by a system that adjusts difficulty based on performance. This method showed better quality in generated research reports on tests, suggesting joint learning of evaluation and creation helps open-ended research tasks.
Large Language ModelsDeep Research Report GenerationReinforcement LearningEvaluatorSolverCo-evolutionary TrainingSelf-evolving FrameworkMeta-harnessOpen-ended Tasks
Authors
Han Zhu, Chengkun Cai, Yuanfeng Song, Xing Chen, Sirui Han, Yike Guo
Abstract
Large Language Models (LLMs) have become increasingly adopted in daily applications, with deep research standing out as a particularly important capability. Unlike traditional question-answering (QA) tasks, deep research report generation lacks definitive ground-truth, making reward design inherently unverifiable and limiting effective reinforcement learning. Existing approaches mitigate this challenge with LLM-as-a-judge and query-dependent evaluation rubrics, but they still rely on static evaluators that cannot adapt their standards as the solver improves, leading to insufficient and eventually saturated optimization pressure. We address this limitation with a \textbf{s}elf-evolving \textbf{co}-evolutionary training framework for deep \textbf{re}search evaluation and generation (SCORE), which tightly couples an evaluator and a solver in a shared-parameter learning process. Rather than treating generation and evaluation as isolated modules, we leverage their intrinsic connection to enable joint improvement within a single shared-parameter model. To restrict this process, we introduce a meta-harness, which dynamically controls the evaluation environment based on solver performance, encouraging valid evaluation dimensions and sufficiently deep evaluator search. Extensive experiments on deep research benchmarks demonstrate consistent improvement in report generation quality, showing that co-evolving evaluation and generation is a promising direction for training open-ended research agents.