Heterogeneous self-play improves realistic traffic behavior without demonstrations
SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations
Machine LearningArtificial IntelligenceRobotics
Summary
Modeling how cars, bikes, and pedestrians behave together is important but hard because they act differently. The authors show that realistic behaviors can emerge by having these road users learn together through self-play, without needing real human examples. Their method, SHRED, uses different rules for each type of road user and creates many scenarios, letting diverse behaviors appear naturally. This approach works well in simulations and improves planning for interactions in traffic.
What this means in practice
- •For autonomous vehicle developers: Train multi-agent policies for cars, pedestrians, and cyclists without relying on costly human driving data.
- •For traffic simulation engineers: Create more realistic and interactive traffic scenarios by jointly simulating diverse road users through procedural self-play.
Authors
Wei-Jer Chang, Zhouchonghao Wu, Akshay Rangesh, Weixin Li, Masayoshi Tomizuka, Wei Zhan
Abstract
Realistic traffic simulation and autonomous planning both depend on modeling interactions among heterogeneous road users whose dynamics, constraints, and behaviors differ fundamentally. Existing approaches predominantly learn realistic interaction behavior from large-scale human driving demonstrations. In this work, we ask whether such behavior can instead emerge purely through self-play. We present SHRED, a framework that jointly trains vehicles, pedestrians, and cyclists through large-scale heterogeneous self-play from random initialization, without demonstrations or logged-trajectory replay. SHRED combines type-specific dynamics and objectives with procedural scenario generation and per-agent randomization, allowing diverse behaviors to emerge through closed-loop interaction. On the Waymo Open Sim Agents Challenge (WOSAC), SHRED jointly controls all three road-user types and outperforms both the closest demonstration-free self-play baseline and prior self-play methods that use demonstration-trained reference policies. Policies trained only on nuPlan map geometry retain comparable realism under zero-shot evaluation on Waymo. For planning, SHRED outperforms all compared planners on the interaction-heavy InterPlan benchmark. Controlled ablations show that heterogeneity in self-play improves both traffic realism and planning performance, with particularly large planning gains in interaction-heavy scenarios. Together, these results establish heterogeneous procedural self-play as a scalable, demonstration-free paradigm for learning realistic and interactive traffic behavior.