JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
2026-08-10 • Robotics
Robotics
AI summaryⓘ
The authors propose JEPA-WAM, a new way for robots to predict future states and generate actions together using a shared model. Unlike older methods that separate prediction and action or simplify future predictions too much, JEPA-WAM keeps detailed spatial information and links current and future observations. This helps the robot better understand the task and improves action prediction from its internal representations. Tests show JEPA-WAM performs very well on challenging robot control tasks and generalizes to different environments and setups.
robot controlworld action models (WAMs)latent spaceV-JEPApredictive modelingaction generationspatial correspondenceLIBERO-PlusRoboTwin 2.0bimanual manipulation
Authors
Yihan Lin, Jiawei He, Shifeng Bao, Chen Zhao, Yang Li, Xiaobo Wang, Yan Wang, Cheng Chi, Jing Zhang
Abstract
Robust robot control benefits from explicitly modeling state transitions, but video-generation world action models (WAMs) introduce substantial deployment cost. Existing latent WAMs avoid explicit future generation, but often compress predictive representations or separate predictive modeling from the representations used for action generation. We introduce JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor. JEPA-WAM predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, while preserving dense patch-level correspondence. Through the shared predictor, transition supervision directly shapes the backbone, from which dedicated representations are extracted for action prediction. The same design can also be instantiated in pretrained VLA policies while preserving their original perception and action pathways. On LIBERO-Plus, JEPA-WAM achieves 79.2%, the best result without large-scale robot-policy pretraining, while its pretrained $π_{0.5}$ instantiation reaches 86.3%, achieving the best overall performance. Experiments on RoboTwin 2.0 and real-world bimanual manipulation further demonstrate strong generalization under visual and spatial shifts.