Robot control model cuts delay twentyfold while keeping accuracy high
W2A: Distilling World Priors into Action Models for Real-Time Robotic Control
Robotics
Summary
Robots need to decide what actions to take quickly to work well in real time. The paper shows a way to train a fast, simple robot action model by learning from a more complex video prediction system that imagines future scenes. The new model can act much faster during use because it doesn’t have to make these predictions anymore. Tests in simulations and with real robots show it still succeeds just as often but runs about twenty times faster.
What this means in practice
- •For robotics developers: Build robot controllers that make decisions promptly for manipulation tasks by using distilled action models without expensive video-based future predictions.
- •For industrial automation teams: Deploy faster and more efficient robotic arms in manufacturing settings that maintain high success rates but reduce computational costs significantly.
Authors
Xiaopeng Zhang, Yongjin Mu, Yueyang Weng, Chengzhi Zhao, Yanjie Li
Abstract
World Action Models (WAMs) have recently shown great promise in robotic control, benefiting from the future imagination capability of large-scale pretrained video models. However, most existing WAMs require explicitly imagining future states at test time, incurring substantial computational overhead and latency that make them unsuitable for real-time robotic control. In this paper, we propose W2A, a two-stage distillation framework for real-time robotic manipulation that distills intermediate denoising representations from a video diffusion model into an action model during training, while entirely discarding the video model at test time. We hypothesize that although video latent representations are high-dimensional, a compressed low-dimensional subspace is sufficient to provide effective supervision for action generation. This makes it possible to transfer such information effectively into an action model. Extensive experiments in both simulation and real-world settings demonstrate competitive success rates while significantly improving inference efficiency. On LIBERO, W2A achieves a 95.9% average success rate. More importantly, W2A achieves about 21x faster inference than the representative WAM baseline.