MoWAM improves robot action prediction with efficient future motion modeling
MoWAM: Explicit Future Motion Prediction for Efficient World Action Models
Robotics
Summary
Predicting what a robot will do next helps robots learn better, but creating full future videos is slow. The authors developed MoWAM, which predicts just the robot’s future movements instead of full videos, making it faster and more efficient. MoWAM uses a special type of AI model to learn and predict future robot motions explicitly, improving how well robots perform and adapt, even in new situations. Testing showed it works well both in simulations and real-world tasks.
What this means in practice
- •For robotics engineers: Design robot control systems that predict explicit future robot motions for faster and more reliable task execution.
- •For automation system integrators: Implement motion-based prediction to efficiently scale and verify robot actions in real-world manipulation tasks.
Authors
Jiayu Wang, Bin Zhu, Yue Yu, Jingjing Chen
Abstract
World Action Models (WAMs) improve robot policy learning by incorporating future dynamics, yet explicitly generating future videos at inference introduces substantial computational overhead. Removing future generation improves efficiency, but leaves future dynamics only implicitly encoded in observation features, which can limit robustness under distribution shifts. We propose MoWAM, an efficient WAM that replaces future video generation with explicit future motion prediction. Instead of reconstructing the complete future scene, MoWAM models structured robot motion as a compact abstraction of the future, capturing how the robot is expected to evolve under the current scene and interaction constraints. A Mixture-of-Transformer architecture learns future visual dynamics during training while jointly predicting motion and action, allowing video generation to be removed entirely at inference while retaining an explicit representation of the future. The compact motion representation further enables efficient inference-time scaling by sampling multiple candidates of motion and action pairs and selecting among them with a motion-aware task-progress verifier. Experiments on LIBERO, LIBERO-Plus, and real-world manipulation tasks demonstrate that MoWAM achieves strong in-distribution performance, improved out-of-distribution robustness, and higher average real-world success than representative WAM baselines. In addition, performance improves as more candidates are explored, demonstrating that explicit future motion provides an effective and efficient basis for inference-time scaling.