World models enable robots to share actions across different bodies
Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors
RoboticsComputer Vision and Pattern RecognitionMachine Learning
Summary
Robots come in many shapes and move in different ways, which makes teaching them new skills complicated. The authors created a method that helps robots learn common patterns of movement and action, regardless of their shape or how fast they move. By first learning shared movement patterns from lots of varied robot data, the method then uses this knowledge to plan actions based on what the robot sees or is told. This means robots with different designs can share knowledge and perform tasks more flexibly.
What this means in practice
- •For robotics engineers: Generate robot actions that work across different robot designs and control speeds using shared learned motion patterns.
- •For automation system integrators: Integrate various robot types into a unified system that leverages common motion knowledge for smoother task execution.
Authors
Liangyu Li, Qingwen Liu
Abstract
We introduce Continuous-Action World Models (CoAWM), a framework for learning cross-embodiment spatiotemporal dynamics priors that support robot action generation across embodiments and execution frequencies. At its core is a shared latent function space that captures the spatial and temporal structure of robot actions. We learn this space through two stages. First, function pretraining brings trajectories from different embodiments and sampling frequencies into a common representation, learning shared dynamics priors through action reconstruction and consistency across temporal sampling views. By accommodating heterogeneous robot trajectories, this stage broadens the range of data sources available for pretraining and facilitates scaling to larger and more diverse datasets. Second, we connect the pretrained function space to a world-model backbone, enabling observations, language instructions, and the current robot state to condition the prediction of a latent action function. A single decoder takes any query time and returns the corresponding end-effector action. By separating shared dynamics priors from task-conditioned action selection and temporal sampling, our approach provides a unified foundation for transferring robot experience across embodiments and control frequencies.