Video anomaly detection improves with guided observation and semantic feedback
Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis
Computer Vision and Pattern Recognition
Summary
Detecting unusual events in videos is tricky because you want to spot what’s abnormal without training a special model. The authors introduce a way to use existing AI models that understand both images and language to look closely at suspicious parts of a video. Their method describes the unusual events in words and then uses that description to improve the detection of anomalies in the video. They tested this on crime and violence video datasets and showed that their approach helped catch anomalies more accurately without needing to retrain the AI.
What this means in practice
- •For security monitoring teams: Improve automatic detection of suspicious events in surveillance videos without retraining detection models.
- •For video content moderation teams: Enhance identification and localization of violent or abnormal content using semantic descriptions to guide review processes.
Authors
Hangil Park, Seungheon Kim, Tae-Kyun Kim
Abstract
Training-free language-based video anomaly detection (VAD) uses frozen vision--language models (VLMs) and large language models (LLMs) to localize abnormal events without task-specific model-weight optimization. Existing approaches increasingly exploit broader temporal and semantic context, yet generic context does not necessarily foreground abnormal events or specify how video-level semantics should modify local anomaly scores. We introduce a training-free semantic feedback framework that uses VAD-guided active observation for video anomaly understanding (VAU) and returns the resulting anomaly-focused description to refine temporal detection. Starting from an upstream anomaly-score curve, we adapt reasoning-based iterative frame selection: high-score regions seed the observations, and anomaly scores together with VLM relevance assessments guide subsequent temporal exploration. A frozen video-language model generates a single open-ended anomaly description from the selected frames. The description is returned through two complementary feedback levels: video-level feedback determines whether the score distribution should be suppressed, preserved, or sharpened, with selective visual rechecking, while local feedback retrieves event-related windows and promotes scores only after target-local visual verification. All model weights remain frozen, and feedback is applied once without regenerating the description. On the HIVAU video-summary evaluations of UCF-Crime and XD-Violence, VAD-guided observation improves GPT-rated reasonability, detail, and consistency over the matched CLIP-guided variant. For VAD, semantic feedback improves the re-evaluated upstream detector from 84.36% to 86.25% AUC on UCF-Crime and from 68.07% to 72.58% AP on XD-Violence, where the final AUC is 91.74%.