Hybrid Robustness Verification for Spatio-Temporal Neural Networks
2026-06-08 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors address the challenge of ensuring AI models in important video-based tasks, like action recognition and autonomous driving, are robust against attacks. They point out that prior methods assume attackers can change every frame, which is unrealistic. Instead, they propose a method called Spatio-Temporal Bound Propagation (STBP) that better reflects how attackers typically change only parts of a few frames. Their approach provides more accurate guarantees that the AI models will be robust and does so more efficiently. They also offer a new benchmark, ST-Bench, to help measure robustness in these kinds of video applications.
3D CNNAdversarial robustnessSpatio-temporal constraintsBound propagationVerificationCertified robustnessLp-norm perturbationsVideo adversarial attacksAction recognitionAutonomous driving
Authors
Sherwin Varghese, Matthew Wicker, Alessio Lomuscio
Abstract
With AI increasingly deployed in safety-critical systems, providing formal robustness guarantees for the underlying models is essential. Existing verification methods either rely on overly conservative approximations or incur prohibitive computational costs. For example, the use of lp-norm perturbations in video settings encodes the belief that the adversary can inject noise in every video frame. In practice, adversarial perturbations exhibit structured spatial and temporal correlations, constrained to lower-dimensional, semantically meaningful subspaces. In this work, we study robustness verification of 3D CNNs processing video and volumetric inputs, targeting applications in action recognition (UCF-101), autonomous driving (Udacity), and medical imaging (MedMNIST) exploiting realistic assumptions on adversarial strength by modelling them as spatio-temporal constraints - where the attacker can modify either a subset of frames or patches within a set of consecutive frames. We demonstrate that modelling realistic constraints enables tighter approximations. We introduce Spatio-Temporal Bound Propagation (STBP), a verification framework that computes an exact closed-form characterization of the first convolutional layer and propagates certified bounds through subsequent layers using scalable approximations. Computing the exact closed form provides the tightest bounds for the first convolutional layer. Thus, we utilise approximation methods in the remainder of the network. To spur further progress in this field, we propose ST-Bench, a verification benchmark for autonomous driving and activity recognition, to systematically evaluate verifiable robustness. Compared to existing verification-based approaches, STBP provides stronger robustness guarantees with significantly improved scalability, achieving 1.7x higher certified robust accuracy under identical perturbation budgets.