Efficient action experts speed up vision language robot control

MambaAct: Efficient Mamba Action Experts for Flow-Matching Vision-Language-Action Models

RoboticsComputer Vision and Pattern Recognition

Summary

Vision-language-action models help robots understand their environment and decide how to move. But some of these models take too long to think, making them hard to use in real-time. The authors present MambaAct, a faster way for robots to decide their actions by making the action part of the model smaller and quicker while keeping the understanding part the same. This new method works well on tests and in real robots, making decisions faster and slightly improving success. The authors also made their code available for others to use.

What this means in practice

  • For robotics engineers: Deploy faster, more efficient vision-language models for real-time robot manipulation tasks with reduced computational cost.
  • For embedded system developers: Integrate compact action experts to speed up on-device decision-making in robots with limited processing power.

Authors

Khuyen Pham, Tran Nguyen Le, Edoardo Fazzari, Cesare Stefanini

Abstract

Vision-language-action (VLA) models have shown strong promise for generalist robotic manipulation, but their inference cost remains a major bottleneck for real-time control. This limitation is especially pronounced in flow-matching VLAs such as $π_{0.5}$, where the action expert is evaluated repeatedly during iterative denoising. We introduce MambaAct, an efficient Mamba action expert for flow-matching VLA policies. MambaAct retains the pretrained visual-language prefix encoder of $π_{0.5}$, while replacing its transformer-based action expert with a compact Mamba-based action expert. Within this expert, Mamba serves as the action mixer that models dependencies across the noisy continuous-action sequence. To preserve visual-language conditioning without repeated token-level cross-attention, MambaAct uses a lightweight Context Compressor to map Vision-language model (VLM) prefix tokens into a compact conditioning signal that gates and modulates each Mamba action block. MambaAct uses a $10\times$ smaller action expert and reduces flow-matching denoising latency by $64.8\%$ on LIBERO and $58.8\%$ in real-world deployment, while improving mean LIBERO success by $1.08$ percentage points across three independent fine-tuning seeds and maintaining gains on LIBERO-Plus and real-world benchmarks. Our code and model weights are publicly available at https://github.com/giakhuyendihoc/MambaAct.