Efficient zero-shot video anomaly detection improves speed and accuracy
Efficient Zero-Shot Video Anomaly Detection via Signal-Aware Adaptive Fusion (SAAF)
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Detecting unusual events in video streams usually requires a lot of computing power, which makes it hard to analyze large amounts of footage in real time. The authors propose a new method called SAAF that smartly selects and analyzes frames based on sudden changes and gradual shifts in the video, combining simple checks before more detailed ones to save time. Their method does not need any training data and still performs as well as or better than some methods that require training, all while running much faster. This can help systems watch video more effectively to spot problems or unusual behavior.
What this means in practice
- •For video surveillance teams: Implement faster anomaly detection that balances speed and accuracy without needing labeled training data on large-scale video feeds.
- •For security system developers: Build security monitoring products that improve real-time detection efficiency using adaptive frame selection and cascaded analysis.$Commercial implications: Enables more efficient and accurate video anomaly detection products for security vendors, reducing operational costs and improving response times.
Authors
Khaled Waleed Dawoud, Muhammad Haris Khan
Abstract
Deploying Vision-Language Models for video anomaly detection confronts a fundamental constraint. Their semantic reasoning demands computational budgets incompatible with the scale of real-world video streams. We present Signal-Aware Adaptive Fusion (SAAF), a framework that achieves efficient zero-shot anomaly detection through three key innovations. First, we introduce a dual-signal adaptive sampler operating in semantic embedding space: a delta signal captures sudden events via frame-to-frame cosine distance, while a drift signal detects static and gradual anomalies through deviation from rolling temporal context. Their OR-gated combination with adaptive EMA-based thresholding provides comprehensive anomaly coverage without training data. Second, we propose a threshold-gated cascaded architecture where lightweight frame-level scoring precedes expensive video-level temporal analysis, invoking it only when confidence warrants additional context. Third, we design a signal-aware weighted fusion mechanism that dynamically adapts score combination based on which signal triggered frame selection. Across four benchmarks, SAAF delivers competitive accuracy at substantially lower cost. Among training-free zero-shot methods, SAAF sets a new state of the art on XD-Violence (92.07%), MSAD (86.70%), and ShanghaiTech (80.54%) AUC-ROC, with XD-Violence and MSAD also surpassing supervised baselines. SAAF sustains 44.95, 48.61, and 41.06 FPS on XD-Violence, UCF-Crime, and MSAD, a 2.43$\times$, 2.63$\times$, and 2.22$\times$ throughput improvement over the prior best zero-shot method. Code: https://github.com/kdawoud91/SAAF