Importance Sampling and PCA for Finding Failures in Commercial Autonomous Vehicles
2026-07-20 • Robotics
Robotics
AI summaryⓘ
The authors tested two special algorithms to find rare failure cases in a commercial self-driving truck system, which are usually hard to discover with normal testing methods. One algorithm uses reinforcement learning to search for likely failure paths, while the other generates diverse failure examples using a diffusion model. Both found crashes during tricky driving maneuvers that traditional methods missed. They also developed a technique using principal component analysis to group these failures and pinpoint critical moments, helping to better understand and diagnose problems in the vehicle's perception system.
Autonomous systemsRare-event discoveryAdaptive stress testing (AST)Reinforcement learningDiffusion modelMonte Carlo simulationPrincipal component analysis (PCA)Failure diagnosisPerception flawsCommercial autonomous trucks
Authors
Hailey Warner, Duncan Eddy, Shreya Parjan, Caroline Cahilly, Harrison Delecki, Matthias Kleinstauber, Chaitanya Shinde, Jerry Lopez, Mykel J. Kochenderfer
Abstract
Methods for discovering rare failures in autonomous systems have so far been demonstrated almost exclusively in simulations with simple, academic driving stacks, leaving open whether they generalize to the more robust planners used in commercial systems. We address this gap by applying two rare-event discovery algorithms to a commercial autonomous trucking stack. Adaptive stress testing (AST) uses reinforcement learning to search for the most likely noise trajectories leading to a simulated collision, while diffusion-based failure sampling (DiFS) trains a denoising diffusion model to sample a diverse set of failures. We show that both algorithms find simulated collisions during merge and cut-in maneuvers where traditional Monte Carlo simulation does not. To make these failures actionable, we introduce a statistical analysis based on principal component analysis (PCA) that classifies failures into common modes and identifies the timesteps that most influence the outcome. We cluster the principal components and invert the PCA transform to recover generalized noise trajectories, and show that these trajectories reproduce failures in identical and similar scenarios. This provides a path from failure discovery to systematic diagnosis of perception-level flaws.