World action model improves robot manipulation by tracking scene changes
CoRe-WAM: Correspondence-Aligned Temporal Residuals for World Action Models
Robotics
Summary
Robots need to compare what they see now to what they saw before to understand how things have changed in their environment. The authors introduced CoRe-WAM, a model that aligns past and current visual information by tracking how objects move between images. This helps the robot focus on meaningful changes instead of confusing different parts of the scene. Their approach improves success rates in robot tasks by using a smart way to combine new and past visual cues without needing to retrain large parts of the model.
What this means in practice
- •For robotics engineers: Enhance robot manipulation systems by integrating temporal correspondences for better scene change understanding.
- •For autonomous system developers: Improve decision-making in autonomous agents by using correspondence-aligned visual changes to inform action selection.
Authors
Bin Zhou, Jialong Liu, Jianan Wang, Changhao Chen, Kani Chen
Abstract
Comparing current and past observations helps robots understand scene changes and select subsequent actions during manipulation. However, comparing visual features at the same image location can mix different scene content when objects or the camera move. We introduce CoRe-WAM, a world-action model that incorporates correspondence-aligned visual changes through a parameter-efficient temporal interface. Its TraceDelta module uses correspondences from a frozen tracking model to transport historical visual features to current locations before computing signed differences in a shared pretrained feature space. Correspondence thus determines which historical content is compared with the present, rather than entering the policy as a separate trajectory representation. A lightweight adapter converts these differences into validity-gated residuals that supplement current visual conditioning, allowing the policy to use recent changes alongside current-scene information. Built on Motus, CoRe-WAM keeps the pretrained backbone weights frozen and optimizes 1.59 million parameters. With a 5,000-update adaptation budget, CoRe-WAM achieves 92.22% clean success across 50 RoboTwin 2.0 tasks, 3.56 percentage points above Motus; on randomized evaluation, it achieves 89.60% success, a 2.58-point gain. Integrating TraceDelta into a StarVLA-based policy improves clean success from 58.10% to 67.62%, supporting transfer of the temporal interface beyond Motus.