Hierarchical world models improve long-horizon robot planning
Hierarchical World Models with Implicit Dynamics
RoboticsArtificial Intelligence
Summary
Planning long and complex tasks is difficult for robots because small mistakes add up and searching for the best steps takes too long. The authors present a new approach called ID-WM that breaks the problem into two levels: a high-level model finds reachable goals, and a low-level model figures out how to get there with precise actions. This helps robots plan longer trips without being stuck by details and improves their ability to handle new challenges they haven't seen before. Their approach showed big improvements in navigation and manipulation tasks compared to existing methods.
What this means in practice
- •For robotics engineers: Develop robots that plan complex navigation and manipulation tasks more reliably in unfamiliar environments by separating high-level goal selection from low-level control.
- •For autonomous vehicle developers: Improve path planning for autonomous vehicles in long and complex routes by combining feasible goal proposals with precise vehicle control actions.
Authors
Gaoyue Zhou, Yvonne Wu, Zichen Jeff Cui, Nicolas Ballas, Mido Assran, Lerrel Pinto, Yann LeCun
Abstract
Planning with world models enables test-time adaptation for embodied agents, offering superior generalization over direct policy learning. However, long-horizon tasks remain hindered by cumulative errors and search complexity. Existing hierarchical methods often use rigid macro-actions or latent skills, leading to brittle representations and limited fine-grained control. We introduce ID-WM (Implicit Dynamics World Models), a hierarchical world model that decouples state-space reachability from local control. Our framework utilizes a high-level model to sample feasible transitions across the state manifold and a low-level world model as an inverse dynamics solver to ground these transitions into precise actions. This allows the planner to identify reachable subgoals without being constrained by low-level execution details. By prioritizing local physical feasibility over global behavioral patterns, ID-WM synthesizes novel long-horizon plans from short segments, transcending the specific behavioral modes seen during training. We demonstrate its effectiveness across three navigation and manipulation suites, achieving a 38.5% improvement over state-of-the-art policy and diffusion-based planners. Crucially, by leveraging this hierarchical structure to mitigate compounding errors, our approach yields a 135% performance gain over single-level world model MPC, particularly in complex, unseen configurations for long-horizon tasks. Videos can be viewed at https://id-wm.github.io