Driving policy behaviors reveal risks under new conditions
Beyond the Leaderboard: Counterfactual Diagnosis of End-to-End and VLA Driving Policies Under Domain Shift
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Current driving AI systems are ranked by how well they perform overall, but these rankings don't explain how or why they behave the way they do, especially in new situations like unfamiliar roads. The authors tested six driving policies by changing real video frames to see if the AI would react correctly—for example, by avoiding a pedestrian or adjusting to nighttime lighting. They found that most systems barely react to pedestrians on their path. Their method reveals how each AI plans, reacts to danger, or overreacts to harmless changes. This new way of testing can help pick safer and more reliable driving systems.
What this means in practice
- •For autonomous vehicle engineers: Evaluate AI driving policies for safe pedestrian responses under novel conditions using edited real-world frames to diagnose behavior beyond leaderboard scores.
- •For simulation software developers: Incorporate counterfactual editing methods to test driving AI plans' sensitivity to pedestrians and lighting changes for better safety validation.
Authors
Ruolin Yang, Zilin Huang, Buoyue Wang, Zhengyang Wan, Yuhao Luo, Zihao Sheng, Sikai Chen
Abstract
End-to-end and vision-language-action (VLA) driving policies are compared by leaderboard rank, but a rank reports an outcome, not the behaviour behind it, so it predicts poorly how a policy will behave at a new site. On six released policies, rank on nuScenes open-loop error or on NAVSIM's leaderboard does not carry over to scenes with a pedestrian near the ego corridor at a new site. We propose a counterfactual check-up: a few hundred real frames, each edited two ways (pedestrian removed, or re-lit by a night-style perturbation), every edit verified by an independent detector, and the change in the planned trajectory read as a diagnosis rather than a score. From these edits two causal axes are read, and five exams built on them separate what a score merges: how far the policy plans to drive, whether seeing the pedestrian buys safety, whether that response scales with danger, whether the plan moves when nothing requires it, and how much an irrelevant lighting change moves it. On 246 NAVSIM near-pedestrian scenes, in the cells where the pedestrian lies on the planned path only 1.9% of responses are genuine avoidance, and under our open-loop protocol the median clearance change is at most 0.03 m and the median change in planned distance at most 0.08 m for every policy. In a pre-registered test from left- to right-hand drive, the exposure and specificity orderings, the lighting verdict and the collision outcome transfer, while point values and the hazard-sensitivity verdict do not. Read as a selection report, the profiles say which policy is safe because it plans short, which covers a human-like distance without yielding, and which is unsteady under a change that requires no reaction, and they price each verdict: most settle within a few dozen frames, hazard sensitivity needs hundreds. Code and edited frames will be released.