Quantization conditioned learning improves vision language action policy reliability
Recovering Low-Bit Failures in Vision-Language-Action Policies via Quantization-Conditioned Reinforcement Learning
RoboticsComputer Vision and Pattern Recognition
Summary
Vision-language-action policies use AI to interpret scenes and act, but making them smaller and faster with quantization often hurts their ability to complete tasks. The authors show that existing training methods only partially fix this problem, especially when tasks are long or visuals change. They propose a new training method called Quantization-Conditioned Reinforcement Learning that better uses hints about task progress to recover performance close to full-size models. Their approach also reduces memory and speeds up processing on specific hardware.
What this means in practice
- •For robotics engineers: Improve the efficiency and reliability of embedded AI for robots performing tasks with visual and language input under real-world variations.
- •For edge ai developers: Deploy smaller, faster vision-language-action models on edge devices while retaining near full-precision task performance under changing environments.
Authors
Seongmin Park, Jaewon Lee, Yujun Heo, Yoonsik Kim, Seheon Choi, Hyoung-rock Kim, Jungwook Choi
Abstract
Quantization lowers deployment costs of vision-language-action (VLA) policies but can compromise task completion. Across five VLA models and 4,050 real-world trials, we systematically reveal that quantization-aware training (QAT) generally improves on post-training quantization (PTQ) but leaves substantial gaps to full precision (FP) on long-horizon tasks and under visual and semantic shifts, while its imitation-based objective does not explicitly use rollout success or partial completion. To exploit these outcome signals, we introduce Quantization-Conditioned Reinforcement Learning (QCRL), an offline RL method coupling progress-based advantage estimation with quantization-aware source conditioning. FP rollouts supply reference experience beyond action-level distillation, progress-scaled terminal penalty distinguishes partial completion, and quantization-aware conditioning selects advantageous reference behavior within the quantized policy. We evaluate weight-only INT4 W4A16 on $π_{0.5}$ and GR00T N1.5 and joint weight-activation NVFP4 W4A4 on $π_{0.5}$. Using only in-distribution data, QCRL reaches 99.3% of FP's mean score across three visual-shift conditions for $π_{0.5}$ with INT4 W4A16, alongside 42.9% model-memory savings. With NVFP4 W4A4, it reaches 100.7% of the same FP baseline, alongside a $1.73\times$ speedup using NVFP4/FP8 deployment. Hardware gains are relative to FP16 on Jetson Thor.