Temporal monitoring improves detection of missing 3D objects in driving scenes
TRIM: Temporal Run-Time Monitoring of 3D Object Detection via Activation Statistics
Computer Vision and Pattern RecognitionRobotics
Summary
Detecting 3D objects like cars and pedestrians is crucial for self-driving cars, but sometimes AI systems miss important objects without obvious errors. The authors propose TRIM, a way to watch how the AI's internal signals change over time to spot when it misses objects. TRIM works by looking at short time sequences of small summaries of the AI’s brain activity without changing the AI itself. Tests show that watching these changes over time helps find missed objects better than checking each image alone, and it runs very fast on normal computers.
What this means in practice
- •For autonomous vehicle engineers: Increase safety by detecting when 3D object detectors miss critical objects during driving using temporal activation monitoring.
- •For robotics system integrators: Monitor 3D perception components to catch failures over time without needing to retrain perception models.
Authors
Hakan Yekta Yatbaz, Konstantinos Koufos, Mehrdad Dianati
Abstract
Reliable perception is essential to intelligent transportation systems, yet deep neural networks can silently miss safety-critical objects while continuing to produce apparently plausible outputs. Existing run-time monitors for 3D object detection generally assess each frame independently and therefore cannot exploit the temporal evolution of occlusion, point sparsity, and scene complexity. We present TRIM, a lightweight temporal monitor that detects missed-object failures without modifying or retraining the underlying detector. TRIM summarises intermediate activations using compact channel-wise statistics and processes short sequences of these summaries with a temporal convolutional network. Experiments on NuScenes show that temporal monitoring improves AUROC from 0.686 to 0.704 and reduces the false alarm rate at 95% recall from 0.818 to 0.759 compared with single-frame monitoring. Early and intermediate backbone features provide the strongest monitoring signal, while the temporal inference head requires only 0.159 ms on a CPU.