Streaming-WAM improves asynchronous robot control with faster visual predictions
Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation
Robotics
Summary
Robots need to predict what will happen next when they move and act. But making these predictions can be slow, causing the robot to wait. The authors introduce Streaming-WAM, a way to predict future robot actions by considering what the robot is already committed to doing, so it can start the next steps faster. This approach helps robots work more efficiently without making more mistakes. They tested this on robot tasks and found it almost triples speed while keeping success rates high.
What this means in practice
- •For robotics engineers: Deploy asynchronous control systems that predict committed robot actions to speed up task completion without losing accuracy.
- •For automation integrators: Integrate Streaming-WAM techniques to reduce waiting time in complex manipulation tasks for industrial robots.
Authors
Xuyao Huang, Yixuan Wang, Zengyao Ye, Boyuan Zhao, Chenyang Yu, Haoran Wen, Zhijie Deng
Abstract
World action models (WAMs) that use future visual prediction at inference time incur substantial generation costs. Asynchronous execution reduces waiting by overlapping inference with robot motion, but visual predictions used for subsequent action generation must anticipate the effects of actions already scheduled for execution during inference. We introduce Streaming-WAM, which couples action-conditioned world modeling with asynchronous robot control to account for committed actions in future visual prediction. At each streaming update, the model conditions future visual prediction on the latest observation and the committed actions, which form the fixed prefix of the next action chunk. The resulting action-conditioned visual features guide generation of the remaining actions within the same joint update, so the continuation is informed by the scene changes expected during execution of the fixed prefix. On LIBERO, Streaming-WAM achieves an average success rate of 98.35\% and reduces mean episode time by a factor of 2.93 relative to Fast-WAM. On the real-world Stamp Paper task, mean episode time falls from 90 s with synchronous Joint-WAM to 38 s with Streaming-WAM. These results show that Streaming-WAM supports efficient asynchronous control while maintaining high task success rates.