World model planners adapt to controller differences for better driving plans
Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift
Robotics
Summary
Planning systems in self-driving cars imagine what might happen when following a path, but the actual car controller can follow that path differently, causing a mismatch. The authors show that simply guessing how the car will move doesn’t help much. Instead, they developed ExecutionBridge, which learns how the controller changes the path and adjusts the planner’s expectations without altering the original plan. This approach improves planning accuracy, especially in tricky driving situations where the car’s actual movements differ more from the planned ones.
What this means in practice
- •For autonomous vehicle engineers: Improve trajectory evaluation by adapting plans to different vehicle controllers without retraining world models.
- •For robotics control teams: Enhance planners’ decision-making by bridging the gap between planned and executed robot motions in systems using fixed models.
Authors
Danqi Zhao, Keyu Chen, Wenchao Sun, Minglei Huang, Jianhe Zhang, Yan Zhu, Hao Cheng, Zifan Cao, Sifa Zheng
Abstract
World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship learned during training. At deployment, a different downstream controller may execute the same planned trajectory differently, creating a planning-execution gap between the frozen model's imagined future and actual execution. A natural remedy is to replace the planned trajectory with its predicted or observed execution. However, we find that better execution fidelity does not necessarily yield better decisions: directly substituting predicted execution trajectories yields only a 0.048 PDMS improvement across 10 unseen controller configurations, far smaller than the gain from our proposed adaptation. This suggests a semantic mismatch between a planned trajectory, which serves as a reference for downstream control, and an executed trajectory, which already describes the resulting post-controller motion. We introduce ExecutionBridge, which infers controller-specific execution characteristics from planned-executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the original planned trajectory unchanged. The planner, world model, and reward head remain frozen. On NAVSIM, ExecutionBridge consistently improves planning across unseen controller configurations and multiple frozen world-model planners, with substantially larger gains in challenging driving scenes where execution mismatch more strongly affects candidate evaluation.