Adaptive question answering improves police body camera video captioning
BodyCam-VQA: Enhanced Body-Worn Camera Video Captioning via Multimodal Reasoning and Probe Question Generation
Computer Vision and Pattern RecognitionComputation and Language
Summary
Police body cameras record important interactions, but the videos can be hard to understand because they are often blurry, noisy, and full of fast actions. The authors created a system that asks detailed questions about what's happening in the video to find important details that usual methods miss. This helps make clearer and more accurate descriptions of the events, which can protect both police officers and civilians. They tested different ways to generate these questions to see which work best.
What this means in practice
- •For law enforcement agencies: Automatically generate detailed and reliable descriptions of body camera footage to assist investigations and accountability.
- •For security system developers: Build AI tools that extract critical forensic details from noisy, real-world surveillance videos using adaptive questioning techniques.
Authors
Karish Gupta, Matthew Alex, Alex Li, Yang Wu, Yun-Wei Chu, Kashif Munir, Xiaotian Zhou, Zhengping Ji, Xiaozhong Liu
Abstract
Police body-worn camera (BWC) footage has emerged as a critical aspect of law enforcement that ensures legal transparency, officer accountability, and the protection of civil rights. However, effectively processing this data remains a significant challenge due to its multimodal video format. BWC videos, in many cases, comprise chaotic scenes with low visual quality, rapid movement/interactions, and high-noise audio that make visual understanding a challenge for even SOTA multimodal models. Current Vision-Language Models (VLMs) frequently overlook critical forensic details, such as the presence of valuable evidence or the latent nuances of suspect-officer interactions, which are vital for fair legal outcomes and civilian/officer safety. To address these limitations, we propose an Adaptive Visual Question Answering (VQA) framework engineered for high-stakes law enforcement. Our framework employs a structured reasoning approach to extract fine-grained visual evidence that traditional captioning systems fail to capture. We experiment with multiple question generation models, including foundation models and fine-tuned open-weight models, to observe performance variation among question generation model implementations. Our results demonstrate that this VQA-driven architecture provides a more reliable, objective, and detailed record of enforcement events, ultimately serving as a powerful tool to protect both law enforcement officers and the public through AI-assisted forensic clarity.