Planning shaped future predictions improve autonomous driving decisions
PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving
RoboticsMachine Learning
Summary
Autonomous cars need to predict what will happen next to plan their moves safely. The authors created a way for the car’s system to focus on predicting future information that is most helpful for planning the car’s path. Their method compresses past observations and uses training tricks to teach the car to imagine useful future scenes. This helps the car make better decisions about where to drive next. Tests show their approach leads to improved driving planning in realistic simulations.
What this means in practice
- •For autonomous vehicle developers: Design car software that predicts future driving scenes focused on planning needs to improve trajectory choices in real time.
- •For robotic navigation engineers: Implement world models that shape future predictions specifically to guide planning, enhancing navigation in dynamic environments.
Authors
Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei, Yongzhi Liu, Sunan Zhang, Jirao Wang, Chen Lv, Bingbing Li, Guodong Yin, Weichao Zhuang
Abstract
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.