Unified score matching improves video anomaly detection and anticipation
A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation
Computer Vision and Pattern Recognition
Summary
Detecting unusual events in videos is important for safety, but current methods struggle with some local details and predicting problems before they happen. The authors propose a single method called Uni-DSM that uses a score-matching approach to better understand normal video patterns and spot when something unusual occurs or is about to occur. Their method shares parts of the model for both finding and anticipating anomalies, overcoming previous limitations that needed future video frames to learn. Tests on several video datasets show their method performs better and faster than existing ones for both detecting and predicting unexpected events.
What this means in practice
- •For video security teams: Deploy models that detect and anticipate unusual events in surveillance footage using a unified architecture for faster and more accurate responses.
- •For smart city infrastructure teams: Integrate video-based anomaly anticipation to proactively handle traffic or public safety incidents before they escalate.
Authors
Congqi Cao, Zhenhe Liang, Hanwen Zhang, Yifan Zhao, Qinyi Lv, Lingtong Min, Yanning Zhang
Abstract
Video anomaly detection (VAD) is a fundamental and safety-critical task in computer vision. Recent generative approaches detect anomalies from a distributional perspective, but remain limited by local anomaly modes. Meanwhile, video anomaly anticipation (VAA), as a proactive extension beyond post-hoc detection, introduces additional challenges. In particular, the contrastive inference paradigm in VAD, which relies on ground-truth frames, is not applicable to VAA, hindering its development. To address these challenges, we propose a unified score-driven framework, termed Uni-DSM, based on denoising score matching (DSM), which models anomaly patterns through likelihood estimation and score functions over the learned data distribution. Within this unified framework, we adopt a shared noise-conditioned score transformer backbone with scene-dependent embeddings and motion-aware weighting for distribution-level modeling. Instead of introducing separate architectures, Uni-DSM unifies VAD and VAA through different inference and supervision paradigms built upon the same score-based formulation. For VAD, we instantiate an autoregressive denoising score matching (ADSM) mechanism, which progressively accumulates anomalous evidence via autoregressive denoising, enabling enhanced perception of local modes beyond visual cues. For VAA, we extend the same architecture by incorporating a lightweight auxiliary decoder and a novel self-distilled denoising score matching (SDSM) mechanism. By constructing supervision from output discrepancies instead of relying on unavailable future ground truth, our method achieves efficient training suitable or early anomaly anticipation. Extensive experiments on multiple benchmark datasets demonstrate state-of-the-art performance in both VAD and VAA while maintaining high efficiency, establishing a unified and scalable pipeline from anomaly detection to anticipation.