Vision-language-action policies refined for precise robot tasks
Beyond Policy Patches: Distilling RL Specialists into Vision-Language-Action Policies
Robotics
Summary
Some robot tasks need very precise movements that general visual and language-guided control systems can't handle well. The authors show how specialist reinforcement learning systems can teach a single unified controller to handle these precision-demanding stages without needing the specialists during actual use. They introduce a method called REPAIR that blends offline and on-the-fly learning by using specialists to suggest better actions only during training. This approach works better and smoother than previous methods when tested on real robot tasks.
What this means in practice
- •For robotics engineers: Create robot controllers that handle precise and long tasks with a single learned policy instead of multiple specialists and switches.
- •For industrial automation teams: Improve robotic arm precision in manufacturing by using training methods that combine specialist skills into more reliable controllers.
Authors
Zhennan Jiang, Tingying Xie, Shilong Ji, Shu'ang Yu, Tianxing Zhou, Yuhui Chen, Jiahuan Gao, Kaizhi Xie, Yuanfan Xu, Feng Gao, Yu Wang, Haoran Li, Dongbin Zhao, Chao Yu
Abstract
Vision-language-action (VLA) policies can handle noncritical stages of long-horizon manipulation yet fail during short precision-critical phases. Lightweight reinforcement learning (RL) specialists can address these failures without directly applying RL to the full VLA, but retain auxiliary controllers and switching mechanisms at deployment. We study how to transfer their capabilities into a single flow-matching VLA. Direct imitation of specialist rollouts can reproduce action jitter and leave the VLA unprepared for states encountered during its own execution. We introduce REPAIR, a value-guided distillation framework that combines offline initialization with on-policy refinement. At each critical-phase observation, REPAIR queries the corresponding specialist for multiple action candidates and weights them using its critic to construct a supervision distribution. A shared weighted flow-matching objective trains the VLA first on observations from specialist-assisted rollouts, then on observations collected through its own execution. Specialists provide supervision without taking control during on-policy collection and are removed at deployment. Across six real-robot tasks with 30 evaluation trials per task, REPAIR achieves 100% success on all six tasks, with lower mean end-effector jerk than specialist-assisted execution on every task.