Vision language models improve video anomaly detection in distributed surveillance
Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Detecting unusual events in surveillance videos is hard when data is spread out and labels are unclear. This paper shows how vision-language models can help by checking suspicious video segments after a simple scoring step is trained in many locations without sharing data. They compare two ways of getting feedback from these models—one uses generated text, and the other uses direct scoring without relying on text. The direct scoring approach worked better, giving more accurate detection without extra complicated processing.
What this means in practice
- •For security system engineers: Improve decentralized video surveillance anomaly detection by integrating model feedback that requires less data sharing and simpler verification.
- •For video analytics platform developers: Develop anomaly detection features that verify suspicious video segments efficiently using lightweight federated scoring and direct model confidence signals.
Tested on one dataset.
Authors
Sébastien Thuau, Amira Gran, Siba Haidar, Rachid Chelouah
Abstract
How can vision-language models help video anomaly detection (VAD) when surveillance data remain distributed, weakly labeled, and resource-constrained? Most weakly supervised VAD methods assume centralized training; recent VLM-based extensions further rely on dense inference, generated explanations, or additional adaptation. We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc. We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities. Experiments on UCF-Crime with InternVL3.5-2B and Qwen3-VL-2B-Instruct show that text-generation verification can improve frame-level AUC after diagnostic temporal post-processing, but remains sensitive to prompts, parsers, model choice, and smoothing. In contrast, the logit interface provides a fixed parser-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs, without temporal post-processing in its main configuration. Since suspect segments are updated independently once available, next-token logit feedback provides a simple segment-local alternative to text-generation verification.