AI summaryⓘ
The authors identify two problems in teaching robots to act based on vision and language: their internal understanding doesn't capture changes over different time scales, and poor-quality behavior from past experiences is reused without filtering. They propose a new method called \method[0m that trains the robot to predict both short- and long-term changes and uses a separate value-based system to improve actions by focusing on better outcomes from past runs. This method improves performance in simulations while keeping the robot's ability to quickly generate actions in real time. Overall, the authors show that combining predictive learning with outcome evaluation helps robots act more effectively from vision and language inputs.
vision-language-action policiesbehavior cloningtrajectory alignmentpredictive learningvalue criticN-step advantagesflow matchingrobot action generationmulti-horizon prediction
Authors
Botong Zhao, Fang Yu, Tim, Senhua Zhu, Xinyuan Chen, Yue Lu
Abstract
Direct vision-language-action policies generate continuous robot actions efficiently, but standard behavior cloning leaves two complementary gaps: their representations are not explicitly required to describe how the scene evolves over multiple time scales, and deployment trajectories of unequal quality are often reused without separating useful dynamics from undesirable behavior. We introduce \method, a direct world-action policy that combines outcome-agnostic predictive learning with outcome-aware policy improvement. \method first retains a local fixed-offset JEPA objective and adds trajectory-relative multi-horizon transition alignment at 25%, 50%, 75%, and 100% of the remaining episode. These training-only targets require the current policy representation to preserve both local physical changes and longer-range task progress, without supplying explicit future tokens to the action head. \method then trains an independent distributional value critic on cumulative deployment trajectories, computes action-chunk-aligned $N$-step advantages, and converts them into positive, negative, or null text conditions for a flow-matching actor. Thus, every valid trajectory can teach what physically happened, while the actor is deployed only under the condition associated with relatively better actions. The multi-horizon predictor and critic are removed from online execution, preserving direct action generation from the current observation, language instruction, and proprioception. \redclaim{Across the three simulation benchmarks, \method achieves the strongest overall performance while preserving the direct actor's online execution path.}