Atomic-wam links task reasoning to robot motion prediction and execution
Atomic-WAM: Execution-Grounded Atomic-Motion Reasoning for World Action Models
Robotics
Summary
The paper tackles the problem that task instructions for robots often don’t specify the exact next move needed. The authors created Atomic-WAM, a system that first understands what the robot should do next using language and vision together, then predicts the right next motion to achieve the task. They improved this by training the system with feedback from actual robot executions to make its motion predictions more accurate and successful. This approach led to better task success rates when tested in a real-to-sim robotic environment, even with vague instructions.
What this means in practice
- •For robotics engineers: Improve robots’ ability to infer and perform the right next motion from vague task instructions using vision and language inputs.
- •For automation system developers: Enable automated systems to generate more precise action sequences that align with real-world physical interactions verified by execution feedback.
Authors
Shijun Shi, Zhe Liu, Zhi Hou, Yinan Chen, Tao Zhang, Kai Hu, Yong Liu, Jiangning Zhang
Abstract
World Action Models (WAMs) typically adapt video foundation models to robotic control through joint video and action prediction. However, task-level instructions may leave the next physical interaction underspecified, requiring semantic grounding and state-dependent motion selection. To this end, we introduce Atomic-WAM, an understand-then-execute framework that connects task reasoning to action generation through atomic-motion descriptions. Specifically, a vision-language reasoner describes the next local motion from the task instruction and visual context, guiding a WAM to predict future video and an action chunk. Furthermore, an automatic annotation pipeline curates temporally aligned, contact-enriched supervision from multi-view DROID demonstrations. To make atomic-motion predictions executable by the WAM, we post-train the reasoner with Execution-Grounded GRPO (EG-GRPO). With the WAM fixed, EG-GRPO adapts the reasoner through rewards for motion appropriateness and execution success. Finally, Atomic-WAM achieves 39.6% task success under default instructions and 31.6% under vague instructions in a real-to-sim evaluation on RoboLab. Project page: https://ssj9596.github.io/Atomic-WAM-project