Simulation-guided learning improves autonomous race car adaptation speed
Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Machine LearningRobotics
Summary
Training robots or cars to drive well often takes a lot of real-world practice, which can be risky and slow. The paper presents a method where a computer first learns about driving from different types of computer simulations, then quickly adjusts when driving the real car using only a short amount of practice. This approach helps the car adapt to new race tracks or surfaces faster and with fewer mistakes. The method works not only in simulated environments but also on a real full-size race car.
What this means in practice
- •For autonomous vehicle developers: Implement adaptive control systems that learn from simulations and rapidly improve with minimal real-world driving data.
- •For robotics software engineers: Create robots that can quickly adapt to new environments using simulation-based prior knowledge combined with limited real-world interaction.
Authors
Adrian Remonda, Jiajun Xi, Nicklas Hansen, Marcus Greiff, Thomas Lew, John Talbot, John Subosits, Xiaolong Wang
Abstract
Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.