RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
2026-04-16 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed RAD-2, a new system for self-driving cars that plans safer and smoother routes by combining a generator that suggests possible paths with a discriminator that picks the best ones based on long-term safety and comfort. They improved the way the system learns by using reinforcement learning techniques that consider time consistency and provide better feedback during training. They also created a special simulation environment called BEV-Warp to efficiently test their method. RAD-2 significantly lowered collisions compared to other planners, and real-world tests showed it drives more smoothly and feels safer in busy city traffic.
autonomous drivingmotion planningdiffusion modelsreinforcement learninggenerator-discriminator frameworktrajectory optimizationcredit assignment problemBird's-Eye View (BEV)closed-loop evaluationsimulation environment
Authors
Hao Gao, Shaoyu Chen, Yifan Zhu, Yuehao Song, Wenyu Liu, Qian Zhang, Xinggang Wang
Abstract
High-level autonomous driving requires motion planners capable of modeling multimodal future uncertainties while remaining robust in closed-loop interactions. Although diffusion-based planners are effective at modeling complex trajectory distributions, they often suffer from stochastic instabilities and the lack of corrective negative feedback when trained purely with imitation learning. To address these issues, we propose RAD-2, a unified generator-discriminator framework for closed-loop planning. Specifically, a diffusion-based generator is used to produce diverse trajectory candidates, while an RL-optimized discriminator reranks these candidates according to their long-term driving quality. This decoupled design avoids directly applying sparse scalar rewards to the full high-dimensional trajectory space, thereby improving optimization stability. To further enhance reinforcement learning, we introduce Temporally Consistent Group Relative Policy Optimization, which exploits temporal coherence to alleviate the credit assignment problem. In addition, we propose On-policy Generator Optimization, which converts closed-loop feedback into structured longitudinal optimization signals and progressively shifts the generator toward high-reward trajectory manifolds. To support efficient large-scale training, we introduce BEV-Warp, a high-throughput simulation environment that performs closed-loop evaluation directly in Bird's-Eye View feature space via spatial warping. RAD-2 reduces the collision rate by 56% compared with strong diffusion-based planners. Real-world deployment further demonstrates improved perceived safety and driving smoothness in complex urban traffic.