From Final Artifacts to Trajectories: Retrospective Process Supervision for Evidence-Grounded Long-Form Generation
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors propose RetroGen, a method that helps language models improve by learning from finished expert work like reports and reviews instead of detailed step-by-step data, which is hard to get for open-ended tasks. RetroGen tries to recreate the hidden steps experts might have taken to produce these final works and checks if these steps make sense. The model then trains itself using the successful recreated examples, without needing better or more detailed guidance data. Tests show this approach helps models better understand sources and create accurate, long responses based on evidence.
trajectory datalarge language modelsagentic abilitiesself-improving frameworkretrospective supervisionlatent trajectoriesgroundingfaithful synthesisevidence-seekingpre-training data
Authors
Junjie Huang, Jiarui Qin, Di Yin, Weiwen Liu, Yong Yu, Xing Sun, Weinan Zhang
Abstract
Trajectory data is getting more vital for training large language models for boosting the agentic abilities. Unlike the verifiable domains such as coding or mathematics, scaling trajectory data for open-ended tasks is much more difficult because these tasks lack singular ground truth and are costly to annotate or verify. In this paper, we propose RetroGen, a self-improving framework of retrospective process supervision. Our key observation is that although expert trajectories are scarce, high-quality final artifacts such as literature reviews, analyst reports and legal judgments, are abundant in pre-training data and can be viewed as compressed traces of the evidence-seeking processes that produced them. RetroGen reconstructs candidate latent trajectories from expert artifacts, verifies them against both the artifact and supporting evidence, and trains models on their own successful reconstruction data, without requiring trajectory data from stronger models. Experiments show that RetroGen improves grounding, faithful synthesis, and long-form evidence-seeking agent tasks.