GraphForge builds real file workspaces to train smarter agents
GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis
Computation and Language
Summary
Training AI agents that can handle real-world tasks needs lots of realistic and verifiable examples. The authors created GraphForge, a system that builds task setups using real files linked by graphs to clearly define and check what needs to be done. This approach helps train agents better, as shown by improvements on multiple AI benchmarks when using GraphForge data. It also uses a process to fix tasks that don’t work well before training happens, making the data more reliable.
What this means in practice
- •For ai tool developers: Create realistic training tasks with verifiable criteria grounded in real file relationships to improve agent coordination and task execution.
- •For automation engineers: Use evidence-anchored tasks to refine AI workflows that involve interacting with diverse files and tools to ensure quality outputs.
Authors
Qisheng Su, Hanchen Wang, Guanru Zhu, Huicheng Jiang, Qiuyinzhe Zhang, Kou Shi, Zhen Fang, Ziao Zhang, Qingnan Ren, Zehui Chen, Tao Gui, Feng Zhao
Abstract
Working agents need to read diverse files, coordinate tools, and produce deliverables. Training such agents requires tasks built on many real files with verifiable results, but few pipelines exist to synthesize this kind of data. Existing pipelines either generate files with models, which lack realism and diversity, or build tasks on real files without task-specific verifiers, leaving result quality unchecked. We introduce GraphForge, an evidence-graph based framework that grounds both the task and its verification in real files. Starting from occupation-grounded seeds for controlled diversity, GraphForge assembles a workspace of real files for each seed and builds an evidence graph over their relations. Since the task statement and rubrics are both derived from this graph, task requirements are backed by the workspace files and each criterion is anchored to the files needed to verify it. An initial rollout further tests executability, and a revision agent repairs the task and rubrics against the original files before trajectories are collected. Fine-tuning Qwen3.6-27B on 2,169 GraphForge trajectories brings GDPVal to 1445.7 (+65.7) under OpenHands, and Workspace-Bench-Lite and SpreadsheetBench II to 63.7 (+7.7) and 24.0 (+13.7) under Claude Code. Rejection fine-tuning on the SFT model's own rollouts, with candidates selected by the evidence-anchored rubrics, yields further improvements on all three benchmarks, suggesting that the rubrics provide a useful selection signal. The data and models are available.