MotionWeave improves robot arm action predictions with motion focus
MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
RoboticsComputer Vision and Pattern RecognitionMachine Learning
Summary
Robots often need to understand how their actions change what they see to perform tasks well. The researchers created MotionWeave, a method that helps robots focus on the important movements their actions cause rather than the whole picture. This improves the robot’s ability to predict what happens next and better plan its actions, especially when it needs to keep interacting with objects over time. Tested on robot-arm tasks, it showed higher success in completing tasks than previous methods.
What this means in practice
- •For robotics engineers: Improve robot arm control by focusing predictions on motion caused by actions for more accurate interaction planning.
- •For automation system designers: Design better manufacturing automation by integrating motion-centered models to sustain action sequences with less error.
Authors
Jingqiu Wang, Yan Wang
Abstract
Vision-Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action tokens through a gated residual. During training, robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over π0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.