Run then walk scheduling improves autonomous driving training efficiency
Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving
RoboticsComputer Vision and Pattern Recognition
Summary
Training autonomous cars to drive safely and efficiently is tricky because focusing on speed can cause unsafe behavior, while prioritizing safety too early makes them overly cautious. The authors found two distinct training modes: one that explores fast progress and another that focuses on safety. They propose a method that first lets the car learn to drive faster, then switches to fixing unsafe behavior. This approach speeds up training and improves driving performance compared to mixing both goals from the start.
What this means in practice
- •For autonomous vehicle engineers: Train autonomous driving systems more efficiently by separating progress discovery from safety correction using the run-then-walk scheduling approach.
- •For robotic system developers: Improve training speed and safety of robot navigation policies by applying a staged reward schedule inspired by run-then-walk reinforcement learning.
Authors
Yuqi Ye, Shangkun Sun, Junhong Lin, Jiayi Zhao, Changhao Peng, Wei Zheng, Guoqing Liu, Tiesong Zhao, Wei Gao
Abstract
Recent VLM-based autonomous driving planners adopt GRPO-style reinforcement learning to optimize driving performance. However, existing GRPO recipes either optimize driving efficiency, risking progress-seeking but unsafe behavior, or enforce early safety constraints, leading to overly conservative behavior; both require lengthy training. To solve these problems, we first reveal two distinct RL regimes: a progress regime (Run-GRPO) that aggressively explores high progress, and a safety regime (Walk-GRPO) that restores safety under stable progress. Based on this finding, we propose $\textit{Run-then-Walk}$, a simple yet effective two-stage reward scheduling strategy for GRPO, achieving both better performance and faster convergence. Unlike one-stage RL, which may focus on progress, safety, or a mixture of both within a single training phase, this schedule explicitly separates progress discovery from safety repair. In the $\textit{Run}$ phase, we focus on progress, allowing the policy to escape the conservative bias and discover high-progress modes. In the subsequent $\textit{Walk}$ phase, we introduce endpoint and safety strategy to repair unsafe behaviors from the Run phase. This reversed schedule overcomes the conservatism of Walk-first methods and the unsafe progress-seeking of joint optimization. We validate it with various VLM-based planners on multiple benchmarks: NAVSIMv1, NAVSIMv2, Navhard, and nuScenes. Extensive experiments demonstrate improved driving performance while requiring 40--50\% fewer RL training epochs than the baselines.