Internal data is not crucial for predicting autonomous driving failures
Auditing Latent-Space Monitors for Autonomous Driving
RoboticsArtificial Intelligence
Summary
Failure monitors try to predict when self-driving car systems might make mistakes by looking inside how they work. The authors tested this on two tasks: making online maps and planning driving actions. They found that you can predict these errors well just by using the outputs of the system, without needing to look inside its internal data. This means that just because a system’s hidden information helps predict failure, it does not prove that the hidden data adds useful information beyond what it shows on the outside.
What this means in practice
- •For autonomous vehicle developers: Improve failure detection systems by using only model outputs and driving commands without needing complex internal data access.
- •For automotive safety teams: Design simpler monitoring tools that predict driving model failures effectively from observable outputs for better vehicle safety.
Authors
Nikhil Kamalkumar Advani, Vishwajeet Shivaji Hogale, Saurav Kumar
Abstract
Runtime failure monitors can use a model's internal representations to anticipate failures. We audit this monitoring strategy across two autonomous-driving tasks: online vectorized map generation with LaneSegNet and end-to-end planning with VAD. We find that frame-level errors are predictable at inference in both tasks. For LaneSegNet, a supervised latent probe reaches Area Under the Receiver Operating Characteristic curve (AUROC) 0.780 for high Chamfer error; to our knowledge, this is the first post-hoc frame-level failure monitor for online vectorized map generation. For VAD, a supervised planning-latent probe reaches AUROC 0.868 for mean-ADE failure. Our audit shows that internal access is not necessary for strong failure prediction. A monitor using only LaneSegNet's prediction outputs reaches AUROC 0.825, while for VAD, ego state, driving command, and the planner's predicted trajectory reach 0.924 on the same mean-ADE endpoint. Adding latent features to either baseline yields no statistically resolved improvement. This observation persists across a broad suite of planning failure endpoints, including endpoints whose labels depend on geometry unavailable to the non-latent baseline. Thus, predicting failure from an internal representation does not establish that the representation provides useful information beyond observable inputs and outputs. We propose an evaluation protocol for testing the incremental value of latent access and release our per-frame failure endpoint labels.