Adjoint guidance flow improves vision language action policies efficiently
Adjoint Guidance Flow: Amortized Critic Guidance for VLA Policies
Machine LearningComputer Vision and Pattern RecognitionRobotics
Summary
Vision-Language-Action policies help robots or systems decide what to do based on what they see and understand. Usually, these policies learn by copying behavior but don't always plan for the best long-term outcome. The authors propose Adjoint Guidance Flow, a new method that uses a lightweight network to guide actions toward higher rewards without extra heavy computation during training or use. This method improves performance reliably across different robot tasks while running faster and using fewer resources than previous approaches.
What this means in practice
- •For robotics teams: Enhance robot task execution by integrating a lightweight guidance network that improves decision-making without retraining the entire policy.
- •For autonomous system developers: Deploy faster and memory-efficient policy corrections in vision-language-action systems to boost performance across diverse tasks without extra computational overhead.
Authors
Jeongsol Kim, Youngjun Jun, Kyumin Choi, Youngmin Kim, Seonghyun Jin, Sunwoo Park, Jangho Park, Kwanyoung Kim, Jong Chul Ye
Abstract
Flow-based Vision-Language-Action (VLA) policies are typically trained by behavior cloning and thus do not explicitly optimize long-term task return. Critic guidance steers generation toward higher-value actions, but existing methods differentiate the critic through a one-step surrogate of the sampler and back-propagate a critic ensemble at every flow step. In contrast, here we propose Adjoint Guidance Flow (AGF), which amortizes trajectory-aware critic guidance into a lightweight guidance network while preserving the pretrained VLA policy. Specifically, we formulate critic-guided flow generation as a deterministic optimal control problem, whose optimal guidance is a costate that carries the terminal critic gradient back through the remaining flow, and regress the guidance network onto this costate while keeping both the VLA and critic frozen. This design provides favorable memory and throughput scaling during training, and inference needs one guidance-network forward pass per step, without the critic ensemble, back-propagation, or adjoint computation. Across LIBERO, RoboCasa, and LIBERO-Pro, AGF consistently improves pretrained VLAs, remains competitive with critic-guidance and policy-fine-tuning baselines, and is the most robust method when a single guidance strength is deployed across tasks. Compared with QGF, AGF runs $3.6\times$ faster per guidance step with $7.0\times$ fewer parameters, with comparable and even better performance, showing that critic guidance can be trajectory-aware and lightweight.