Self-play method uses uncertainty to improve multi-agent driving
Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies
RoboticsArtificial IntelligenceMultiagent Systems
Summary
Driving cars that can handle many different situations is hard because they often only learn from limited examples. The authors propose a new training way where some cars try to find tricky driving situations by creating uncertainty in others. This uncertainty helps identify the hardest cases automatically without someone having to design them. Cars learning in this way get better at driving safely in new, unseen scenarios. Their method works better than usual approaches when tested on different driving datasets.
What this means in practice
- •For autonomous vehicle engineers: Train multi-agent driving policies that generalize better to new traffic conditions without manual scenario design.$Commercial implications: Enables safer and more adaptable autonomous driving software that can handle diverse real-world environments.
- •For simulation platform developers: Create automated scenario generators that focus training on challenging driving situations using uncertainty signals.
Authors
Sajad Marvi, Julian Schmidt, Alessandro Canevaro, Julian Jordan, Abhinav Valada
Abstract
Learning robust driving policies that generalize beyond their training domain remains a central challenge in motion planning. We propose an uncertainty-driven self-play scheme, in which half of the agents are rewarded for increasing their neighbors' epistemic uncertainty, while the other half learn to drive safely as the exploring agents continuously seek out their weaknesses. Epistemic uncertainty indicates where the policy is least confident and serves as an automatic proxy for scenario difficulty. Exploring agents use this signal to seek out hard cases, generating a challenging curriculum without manual scenario design. The resulting policy outperforms standard self-play, achieving best-in-class zero-shot transfer to nuplan Val14 (99.12% vs. 97.97%) and Argoverse 2 (82.39% vs. 78.03%) without fine-tuning, while also improving performance over standard self-play on Waymo. Our diversity-driven approach achieves 85% higher LSTM hidden-state coverage than the baseline, indicating a much richer set of behaviors. Further analysis shows that epistemic uncertainty serves as a calibrated difficulty detector, with highest-uncertain agents exhibiting 14.8 times the combined failure rate across collision, off-road, and DNF, of lowest-uncertain agents, enabling automatic prioritization of challenging scenarios without ground-truth labels.