Pretrained models guide progress in long tasks without task-specific training
World Potential Model: Pretrained World Knowledge as Progress Potentials
Artificial Intelligence
Summary
Sometimes, computer programs that need to finish long tasks only get feedback when they finally succeed or fail, which makes it hard for them to learn which steps help. The researchers asked if models already trained with general world knowledge can tell if the program is making meaningful progress at each step. They created a model called the World Potential Model that uses this pretrained knowledge to estimate how close the program is to its goal without extra training for each task. They tested it on simulated environments and found it helps programs learn better by giving step-by-step feedback, not just final success or failure.
What this means in practice
- •For robotics engineers: Enhance robot learning for complex tasks by providing feedback based on pretrained world knowledge to better recognize task progress during long procedures.
- •For video game ai developers: Improve non-player character learning in games by using pretrained models to evaluate progress at intermediate steps, leading to more sophisticated behaviors.
Authors
Jun Zhao, Jixin Tang, Yang Shu, Jinyang Wu, Yuyang Lu, Jingqi Tong, Hao Xu, Weifeng Ge, Qi Zhang
Abstract
Long-horizon language agents often receive supervision only from terminal task outcomes, leaving little signal for distinguishing productive intermediate behavior from stagnation or even regression. Rather than learning a separate value function or process reward model for every task, we ask whether pretrained models can recognize task progress from their existing world knowledge. We formalize this capability with a World Potential Model (WPM), a goal-conditioned evaluator of task-relative realized progress in agent contexts. In ALFWorld and ScienceWorld, off-the-shelf pretrained models substantially outperform chance at recovering realized-progress structure without task-specific evaluator fine-tuning. We further anchor these progress judgments to task-specific milestones to obtain scalar world potentials, whose temporal differences provide process-sensitive step-level credit for policy optimization. Under matched comparisons, WPM-guided optimization improves success over outcome-only GRPO across all evaluated configurations. Together, these results provide initial evidence that pretrained world knowledge can support reusable realized-progress evaluation and provide useful supervision for long-horizon agents.