Summary
Detecting unusual events in videos is helpful for safety but explaining why a system decided an event is strange can be hard, especially if people’s privacy is protected by hiding their identities. The authors created XAVAD, a system that can spot odd events in anonymized videos and explain what type of anomaly happened, where it happened in the scene, and describe the setting. They use advanced AI tools like language and vision models to identify anomalies without training on specific examples and to generate understandable descriptions. Their tests show this system is more accurate and efficient than previous methods while keeping people’s privacy.
What this means in practice
- •For security operations teams: Automatically detect and explain unusual events in anonymized security videos for safer monitoring and faster decisions.
- •For privacy compliance officers: Assess and improve trade-offs between video anomaly explainability and personal privacy protection in surveillance systems.
Abstract
Despite the promising development of deep learning for video anomaly detection (VAD), explainability (XAI) of these models remains a challenge. Explainable and privacy-protecting VAD builds transparency and trust that facilitates adoption of computer vision systems, including safety and security monitoring. However, anomaly explainability in privacy-preserving VAD systems is underexplored. In this study, we present the XAVAD approach, a unified descriptive anomaly XAI for an anonymized VAD, that integrates flexible zero-shot anonymization, anomaly type categorization, visual localization, and scene context description. We provide empirical and qualitative assessments of multifaceted explanations integrated with full-body visual anonymization for privacy protection. We adopt a multimodal CLIP embedding for videos, using improved LLM-assisted prompting and pool-scoring mechanisms for anomaly-type identification. We enhance Grad-ECLIP for spatial anomaly heatmap localization, leveraging computation and accuracy through our novel prompt-guided patching autofocus mechanism. We employ VLMs to generate context and reasoning descriptions of detected anomalies through curated prompt engineering. We have evaluated the accuracy and computational cost of XAVAD on several publicly available CCTV and video datasets. Our experiments demonstrate substantial gains of our XAVAD over state-of-the-art benchmark approaches and provide insights into privacy protection trade-offs in XAI for VAD. We further present ablation studies and assessments across diverse configurations and modeling alternatives to demonstrate the efficacy of our proposed methods. Code: https://github.com/muleina/XAVAD}{https://github.com/muleina/XAVAD .