Verify act and distill improve vision language action model performance
Verify, Act, Distill: Turning Test-Time Compute into Test-Time Adaptation for Vision-Language-Action Models
Robotics
Summary
Vision-language-action models choose actions based on what they see and understand. Usually, these models spend extra computing power at decision time to pick better actions, but this is slow and can't be saved for later use. The authors propose a way for the model to learn from this extra thinking during action, improving itself for future decisions without extra slowdowns. This method raised task success in tests, showing the model can get better over time by learning from its own verification process.
What this means in practice
- •For robotics developers: Enable robots to improve task performance by learning from verification steps done during operation without extra retraining costs.
- •For autonomous system engineers: Build vision-language-action systems that adapt to new or changing environments by distilling verification insights into their policies.
Authors
Kaneyoshi Hiratsuka, Satoshi Yamamori, Jun Morimoto
Abstract
Test-time compute (TTC), which scores multiple candidate actions at inference time, is a powerful way to improve the execution performance of vision-language-action (VLA) models, but existing methods keep the policy frozen, so the gain must be recomputed at every execution. We propose Verify, Act, Distill, a framework that consolidates the gains of TTC into the VLA itself. A fixed verifier scores candidates generated by a flow-matching VLA, the weighted aggregate is executed, and the observations and aggregated chunks from successful adaptation episodes are distilled directly into LoRA parameters with the standard flow-matching loss. This reuses supervision that TTC has already produced during execution, without additional reinforcement learning or rollouts dedicated to teacher generation. On the 40 LIBERO tasks with a SmolVLA 5k checkpoint, distilling from only 10 episodes per task raises the mean success rate of verifier-free single-candidate generation from 54.1% to 70.6%, retaining about 74% of the improvement obtained by aggregating eight candidates. Applying the same aggregation to the distilled policy further raises the success rate from 76.4% to 82.3%, with a particularly large gain on LIBERO-Long (44.0% to 57.8%), and the single-candidate policy also improves on LIBERO-plus, which contains distribution shifts. These results show that using verification as a learning signal, rather than only as an execution-time selector, both retains the improvements of TTC and re-amplifies the effect of subsequent TTC.