Dual latent world model improves long horizon planning for robots
Beyond a single latent space: a dual-latent world model for long-horizon planning
Artificial Intelligence
Summary
Planning far into the future is hard for AI models that imagine what will happen next because small prediction mistakes add up and make it hard to reach distant goals. The authors created a new model called Dual-WM that splits planning into two parts: a high-level plan for long steps and a low-level plan for precise actions. They also developed a way to teach the model to focus on important predictions at different times. This approach helps AI systems succeed more often when completing tasks that need many steps to reach a goal.
What this means in practice
- •For robotics engineers: Improve robots’ ability to plan sequences of actions over long periods to reach complex goals with higher success rates.
- •For autonomous vehicle teams: Enable vehicles to plan multi-step routes more reliably by separating high-level route planning from low-level control commands.
Authors
Delin Zhao, Zhengrong Yue, Shaobin Zhuang, Junlin He, Xiaoyu Chen, Zikang Wang, Yuxin Liu, Limin Wang, Yali Wang
Abstract
Latent world models often struggle with long-horizon planning despite accurate short-term predictions. Recursive rollouts accumulate errors, while distance concentration in high-dimensional latent spaces can weaken goal discrimination. We introduce the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning through distinct state representations and dynamics models. The low-level model predicts action-conditioned transitions, while the high-level model uses learned macro-actions to plan over longer temporal spans. We also propose Long-Horizon Representation Learning with Weighted Rollout (LoRe), which supervises self-generated predictions at both levels. An analysis of recursive error propagation motivates exponential horizon weights with separate decay rates for the two temporal scales. During planning, the high-level model generates latent subgoals that the low-level model refines into actions for precise execution. We evaluate from-scratch Dual-WM on five goal-conditioned visual control tasks against the task-wise strongest baselines without actor-guided proposals. At goal offsets of 50 and 100 environment steps, mean success increases from 75.9% to 84.4% and from 61.4% to 69.5%, respectively. At offset 100, Dual-WM outperforms these baselines on all five tasks and improves mean success over LeWM by 30.8 percentage points. Ablations and supporting analyses provide evidence of more informative representations for goal evaluation and greater consistency under recursive prediction. These results highlight the value of separating temporal roles and training across multiple horizons for reliable latent planning. Our core implementation is available at https://github.com/DeLin1001/Dual-WM-Official.