AI summaryⓘ
The authors studied how important things like traffic lights and road signs are for self-driving cars that learn end-to-end (from seeing to driving). They found that most past work ignored these traffic elements and focused only on moving things like cars and people. To fix this, the authors added detailed traffic element info to different driving datasets and tested many types of driving models by simply including these signals. They saw that including traffic elements consistently made the driving systems better across multiple tests and datasets, showing these elements help autonomous driving work more reliably. Their approach even improved the best methods on a tough driving benchmark.
end-to-end drivingtraffic elementsautonomous drivingtraffic lightsroad signsdriving datasetsperception-prediction-planningvision-language-action modelsregression-based plannersdiffusion-based policies
Authors
Zongzheng Zhang, Jijun Wang, Saining Zhang, Shuo Wang, Yiru Wang, Hai Yang, Yang Chen, Yuwen Heng, Hao Sun, Anqing Jiang, Hao Zhao
Abstract
Traffic elements such as traffic lights and road signs play a fundamental role in human driving decisions and should naturally influence end-to-end driving performance. However, existing end-to-end driving research predominantly focuses on dynamic road participants (e.g., vehicles and pedestrians), while the role of traffic elements remains largely unexplored. The community still lacks a systematic study quantifying their impact, largely because public datasets rarely provide structured traffic-element annotations and modern driving systems vary widely in architecture and training paradigm. In this work, we present the first systematic investigation of traffic element awareness for end-to-end autonomous driving. We construct a unified research infrastructure by augmenting multiple public driving datasets with comprehensive traffic-element annotations. To support diverse model families, we adopt a minimal and universal integration design that incorporates traffic-element signals into existing pipelines in a plug-and-play manner with negligible architectural modification. We evaluate this design across modern paradigms, including perception-prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory-scoring frameworks, on nuScenes, NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Across all paradigms and datasets, this simple integration consistently improves driving performance, demonstrating that traffic element awareness provides a robust and generalizable signal for end-to-end driving systems. Notably, on the challenging NAVSIM-v2 benchmark, our approach significantly improves state-of-the-art architectures and data pipelines, establishing a new state of the art.