Vision language action models learn to recover from mistakes
OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models
Artificial Intelligence
Summary
Robots often learn to copy expert actions exactly when doing tasks, but this can make it hard for them to fix mistakes or handle unexpected situations. The authors introduce OGPO, a way to train robots that focuses on reaching key important states in a task instead of just repeating every single step. This helps robots become better at recovering and finishing tasks even if things don't go exactly as shown in demonstrations. Tests show OGPO improves robotic performance and robustness without needing extra live practice.
What this means in practice
- •For robotics software engineers: Improve robot control software to better handle mistakes by training policies to reach important task states rather than mimic exact actions.
- •For automation system integrators: Deploy more robust robotic systems in manufacturing by using goal-focused policy training to maintain task completion despite unexpected state deviations.
Authors
Xule Gao, Xi Wang, Zehua Zang, Rui Wang, Changwen Zheng, Chuxiong Sun
Abstract
Vision-language-action (VLA) models have shown strong promise in robotic manipulation, yet their post-training still predominantly relies on trajectory-level supervised imitation over robot demonstrations. While effective for learning feasible actions, this paradigm supervises policies to reproduce expert actions at each step, biasing them toward demonstration-specific action paths and providing limited guidance for recovering toward task-progress states once execution deviates from demonstrated trajectories. To address this limitation, we propose OGPO, an Offline Goal-conditioned Policy Optimization framework for recoverable VLA post-training. Instead of treating offline demonstrations merely as step-wise action labels, OGPO adaptively relabels trajectories with semantic key states and constructs goal-conditioned process rewards to optimize the policy for reaching these task-progress states. In this way, OGPO shifts VLA post-training from trajectory-level action imitation to key-state reachability learning, enabling policies to acquire a more robust ability to complete goal states from feasible off-demonstration states without requiring additional online interaction. Experiments on LIBERO and MetaWorld show that OGPO consistently improves VLA policy performance over standard supervised fine-tuning. Moreover, evaluations on LIBERO-Plus and our proposed LIBERO-ReAct, a mid-execution perturbation benchmark, further demonstrate that OGPO achieves stronger robustness and goal-directed recoverability under distribution shifts and recoverable off-demonstration states.