Compact policy learns driving from simulator without collisions
Imitation Learning for Autonomous Driving in CARLA
Artificial IntelligenceRobotics
Summary
Driving a car requires understanding how actions influence what you see next. This paper studies how a small computer program can learn to drive a car in a simulator by looking at past expert driving examples. Their program uses images, laser scans, and car information to decide how to steer, brake, and accelerate. They trained it on a few hours of driving data, and it can drive for extended periods on familiar and new routes without crashing. The authors also made their program and data publicly available for others to try.
What this means in practice
- •For autonomous vehicle developers: Train compact driving models using multimodal offline data for reliable autonomous navigation in urban simulators.
- •For simulation software engineers: Incorporate route generation and verified autopilot data collection mechanisms for enhanced driving data diversity within simulators.
Authors
Jordy Kieto
Abstract
Behavioral cloning trains a policy offline on expert demonstrations, but deployment is closed loop: each action affects the observations the policy receives next. We study how much closed-loop driving competence a compact multimodal policy can acquire from offline demonstrations in the CARLA simulator. The policy uses five-frame histories of RGB images, LiDAR, vehicle telemetry, and lane waypoints to predict throttle, brake, and steering at 20 Hz. Demonstrations were collected in three stages, ending with a systematic route-generation procedure that enumerates spawn points and feasible maneuvers and verifies completed autopilot routes. The released 1.36 million parameter policy was trained on 236,882 windows, representing about 3.3 hours of driving from 448 captures. The resulting policy drives autonomously for hours on training and held-out routes. In our runs, it did so without collisions and also transferred qualitatively to an unseen CARLA town with different road geometry. We also observed recovery from large trajectory deviations, although we do not claim systematic recovery without controlled evaluation. We report offline metrics and distinguish measured results from qualitative closed-loop observations. We release the code, trained checkpoint, ONNX model, data sample, and an evidence audit for the reported claims.