Rolling WAM speeds up robot action planning with partial video prediction
Rolling-WAM: World Action Models with Rolling Imagination
RoboticsArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Robots need to predict future video frames and actions to plan their moves, but doing all this at once takes too long and slows them down. The authors found a way to spread out the work over time, partially improving future predictions while fully preparing the next action. This approach lets the robot update its plans faster while keeping important visual context from before. Tests show this method keeps robot performance good but makes planning over 4 times faster.
What this means in practice
- •For robotic system developers: Speed up robot motion replanning by incrementally refining future action and video predictions.
- •For industrial automation teams: Improve real-time responsiveness of robots handling manipulation tasks by reducing action update latency.
Authors
Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, Yue Wang
Abstract
World Action Models (WAMs) couple action generation with future visual prediction for robotic manipulation. However, completing the joint video-action denoising process at each replanning cycle incurs substantial latency, delaying action updates and limiting closed-loop responsiveness. We present Rolling-WAM, a formulation that distributes joint denoising across successive replanning cycles. Our method maintains a sliding window of video-action chunks at staggered noise levels. At each step, a rolling noise schedule fully denoises the imminent action chunk for execution, while partially refining farther-future chunks. As the window advances with new camera observations, the retained future chunks continue their denoising process. This distributes the computational cost over time while carrying an evolving visual-action context across chunk boundaries. Evaluations on LIBERO, RoboTwin, and a real-world Unitree G1 humanoid show that Rolling-WAM achieves competitive manipulation performance. By removing the need to denoise the entire prediction horizon from scratch, it delivers a 4.5x steady-state replanning speedup over standard joint WAMs.