Region selection improves video anomaly detection and localization

Region-VAD: Video Anomaly Localization and Detection via Region Selection

Computer Vision and Pattern Recognition

Summary

Detecting unusual events in videos is hard because these events often happen in small parts of the scene. Existing methods look at the whole video frame at once, which can confuse the event with the background and not show exactly where the problem is. The authors developed Region-VAD, which picks the part of the video frame that changes the most between moments and looks for anomalies only there. This method helps find and locate unusual events better than before without needing detailed labels.

What this means in practice

  • For video surveillance teams: Use region-based video analysis to pinpoint and detect unusual activities in surveillance footage more precisely without detailed labeling.
  • For security system developers: Incorporate region-focused anomaly detection into security software to improve identification and localization of suspicious events in video streams.$Commercial implications: Enables development of advanced security products that better detect and locate anomalies, appealing to clients needing detailed event monitoring.

Authors

Giacomo D'Amicantonio

Abstract

Anomalies are local events, yet weakly supervised video anomaly detection scores them globally. Most methods pool an entire frame into a single vector, rank frames by that vector, and derive a location afterwards as a saliency map of the score. Full-frame pooling mixes the anomaly with its background, so a detector can reach competitive benchmark scores from scene context without being able to show where in the scene the anomaly occurs. We introduce Region-VAD, which first selects a region and then detects the anomaly from that region alone. Region-VAD selects an explicit rectangle over the tokens of a frozen foundation model and scores anomalousness only from the tokens inside the box, through attention pooling followed by a standard temporal head. The rectangle is not learned, it is the box in which the frozen tokens change most between consecutive segments, in all four feature projections of the encoder, and it is found by exhaustive search over the rectangles of the token grid. Only the pooling and the temporal head are trained, from video-level labels without boxes, masks or frame-level annotations, and no trained component reads a token outside the selected region. Experiments on five datasets, with one localizer configuration shared by all datasets and backbones, show that Region-VAD achieves state-of-the-art anomaly localization on UCF-Crime and MSAD at competitive detection performance.