Structured monitoring improves risk alerts in autonomous driving systems
Structured Representation Probing for Risk Anticipation in End-to-End Autonomous Driving
Machine Learning
Summary
Self-driving cars must avoid accidents by recognizing risky situations early on. The authors designed a tool called Structured Representation Probe (SRP) that looks inside a car’s driving system to find signs of trouble before accidents happen. This tool reads the car’s internal understanding of other vehicles, its own motion, and the road to give early warnings about potential failures. Tests showed it predicts problems better and faster than older methods, and it works even on new driving routes.
What this means in practice
- •For autonomous vehicle engineers: Provide early warnings of potential driving failures by monitoring internal representations of the driving policy without changing its operation.
- •For robotics system integrators: Improve real-time risk detection in robots navigating traffic environments by using structured internal feature monitoring.
Authors
Jiawei Liang, Tianrui Lou, Le Yang, Xianghao Jiao, Wei Wang, Xiaochun Cao
Abstract
End-to-end driving policies form structured internal representations of traffic, ego behavior, and road constraints. These representations offer a monitoring interface for anticipating failures, complementing risk signals from trajectories, controls, and manually designed features. We introduce the Structured Representation Probe (SRP), a lightweight monitor that reads actor, ego, and map representations from a frozen policy. SRP learns a separate linear risk direction for each source and aggregates element scores through cardinality-normalized log-mean-exp pooling, retaining sensitivity to salient evidence while reducing dependence on token count. Event-window supervision links the readout to upcoming hard failures, and auxiliary event type heads jointly train the source directions across failure modes. Online alerting uses a single generic score without modifying the driving policy. Bench2Drive experiments on DriveTransformer, UniAD, and VAD show higher frame-level PR-AUC and ROC-AUC than the evaluated runtime baselines. On DriveTransformer at a horizon of 2 seconds, ROC-AUC reaches 0.833 versus 0.715 for temporal consistency, with 2.67% readout runtime overhead. The trained probes also retain the ability to distinguish impending failures on new routes.