Feedback-aware sampling boosts speed of vision-language-action models
FASA: Feedback-Aware Sampling Adaptation for Efficient Diffusion-Based VLA Models
Robotics
Summary
Vision-language-action models help robots understand and interact with their environment but take a lot of computing power to work in real-time. The authors propose FASA, a system that adjusts how many steps the model uses based on feedback from the robot’s sensors during its task. This adaptive method lets the model work faster without losing accuracy by paying attention to what the robot sees and feels while acting. Their tests show FASA increases processing speed by up to 1.45 times while keeping success rates similar.
What this means in practice
- •For robotics engineers: Implement adaptive step sampling to speed up vision-language-action models on robots performing real-time tasks without retraining.
- •For edge ai developers: Deploy heavy generative embodied AI models efficiently on resource-limited edge devices by dynamically adjusting computational workload during execution.
Authors
Yuchen Han, Jianhan Wu, Xiaoyang Qu, Lingwei Kong, Shiyi Li, Jianzong Wang
Abstract
Diffusion-based Vision-Language-Action (VLA) models achieve strong performance in embodied tasks, but their iterative sampling imposes heavy computational and memory-access cost, blocking real-time deployment on edge platforms. Existing acceleration methods either require expensive training (e.g., distillation, flow matching) or degrade perception via statically scheduled pruning and caching, ignoring the dynamic workload variance of robotic interactions. This paper presents FASA (Feedback-Aware Sampling Adaptation), a training-free runtime framework that treats real-time multimodal feedback as a control signal for the denoising pipeline: an interaction-driven range adaptor modulates the global sampling-step budget based on visual and gripper-force feedback, and a proprioception-aware step adaptor pinpoints the optimized step within the adapted range. This co-designed framework allows the underlying hardware architecture to adaptively match the workload demands of different execution phases. Comparative evaluations across several benchmarks show that the inference speed can be increased by up to 1.45$\times$ while maintaining competitive success rates, providing a novel dynamic runtime architecture paradigm for deploying heavy generative embodied AI workloads onto resource-constrained computing platforms.