Vision language action system measures trust in its stepwise reasoning
How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function
RoboticsComputer Vision and Pattern RecognitionMachine Learning
Summary
Deciding actions from complex reasoning steps can sometimes lead to mistakes in vision-language-action systems. The authors propose a method to adjust how much the system relies on each reasoning step, based on measuring how valuable that adjustment is for the overall plan. By treating these adjustments like a decision problem and learning value estimates, their approach improves decision quality in simulated driving tasks and real-world driving benchmarks. This method led to better performance than previous approaches, suggesting more careful trust in intermediate reasoning helps action policies.
What this means in practice
- •For autonomous vehicle engineers: Improve driving action policies by adaptively tuning trust in intermediate reasoning steps to boost on-road performance and safety.
- •For robotics control teams: Enhance multi-step decision-making in robots by measuring and adjusting the influence of reasoning during action planning to reduce errors.
Authors
Dabin Seo, Eunhyek Joa
Abstract
Reasoning vision-language-action (VLA) policies write a reasoning trace and then decode the action from it. A wrong trace yields a wrong action, and even a correct-looking trace can mislead the plan. Existing remedies ask whether the reasoning is right. We ask how strongly it should steer. CoC-CFG applies classifier-free guidance between the policy's context and the same context without the reasoning span. One weight per Euler step attenuates or amplifies the reasoning, and w=1 recovers the original policy. We treat the choice of this weight as a Markov decision process whose action values are measured, not bootstrapped. With the expert frozen and the noise fixed, applying a weight for one step and finishing the plan under the current controller returns that weight's value exactly. Q-adaptive guidance fits these values with small critics and improves the controller conservatively. In the Alpasim driving simulator it outperforms every fixed weight and online RL (SAC, GRPO) in open loop, and in closed loop it passes 262 of 300 scenes, against 252 for the original policy. On held-out frames of the Waymo end-to-end driving benchmark, its first three stages outperform every fixed weight, four adaptive-guidance methods, SAC and GRPO. On the hidden test set, a frozen Alpamayo 1.5 with rule-based camera and vehicle conversions and Q-adaptive guidance sets a new state of the art among published methods: 8.135159 rater feedback score, against 8.079 for the best published method.