Traffic sign compliance benchmark reveals planner safety gaps
TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving
RoboticsComputer Vision and Pattern Recognition
Summary
Many self-driving car programs score well on overall driving tests but still break important traffic rules, which can be dangerous. The authors created TrafficSignBench, a computer-based test that checks if these programs follow 34 types of traffic signs during driving simulations. This test uses many different road scenes and scenarios to find where planners fail to obey rules. They also show how to improve planners by teaching them to respect traffic signs explicitly.
What this means in practice
- •For autonomous vehicle developers: Assess and improve self-driving car planners’ ability to follow traffic sign rules using a scalable and interpretable test framework.
- •For simulation platform engineers: Integrate a large set of controlled traffic-rule scenarios and automated violation checks into driving simulators.
Authors
Victoria Smirnova, Viktoriia Zinkovich, Gregorii Bukhtuev, Artem Belyaev, Andrey Kuznetsov, Denis Shepelev, Vlad Shakhuro
Abstract
Autonomous driving planners are typically evaluated using aggregate metrics such as driving score, destination rate, and collision rate, which do not explicitly measure compliance with traffic rules. As a result, planners can achieve high benchmark scores while still exhibiting unsafe or illegal behaviors, limiting their applicability to real-world deployment. To address this gap, we introduce TrafficSignBench, a large-scale, traffic sign-centric benchmark for systematic and interpretable evaluation of traffic-rule compliance in autonomous driving. Our framework combines real-map-based simulation for realistic road layouts with rule-targeted procedural scenario generation for scalable and balanced coverage of underrepresented rules. We implement traffic rules corresponding to 34 traffic signs, each equipped with an automatic rule checker for detecting violations during closed-loop execution. This design yields 29,000 diverse road scenes and 29 distinct testing scenario types, enabling controlled evaluation of rule-specific planner behavior. We construct 5,800 testing scenes and demonstrate that current autonomous driving planners can exhibit poor traffic-rule compliance despite strong performance on standard evaluation metrics. To address this limitation, we transform existing planners into rule-compliant trajectory experts via explicit traffic-sign constraints, enabling scalable generation of high-quality oracle trajectories for fine-tuning.