Multimodal cues help predict violence moments before it starts
Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues
Computer Vision and Pattern Recognition
Summary
Figuring out when someone might become violent just before it actually happens is very challenging. This paper looks at short video clips showing behavior right before violent incidents without including the incident itself. The researchers use face appearance, sounds, and body movements to spot risky behavior, and find that using all these together helps predict violence better than any single source alone. Their method reached about 91% accuracy in guessing which clips showed riskier behavior before violence started.
What this means in practice
- •For security system developers: Build surveillance systems that analyze video, audio, and motion data jointly to flag potential violence risk moments before it begins.$Commercial implications: Enables development of preemptive security alert tools for venues and public spaces to reduce violent incidents.
- •For public safety teams: Use multimodal signals to anticipate elevated risk situations in real-time, allowing quicker preventive actions.
Authors
Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz, Sudip Mittal, Shahram Rahimi
Abstract
Detecting violence after it begins is important from recognizing behavioral cues that appear immediately beforehand. This work studies short-horizon pre-incident risk recognition from multimodal video signals. We construct a binary Normal-versus-Risky setting from temporally annotated XD-Violence clips, using 443 samples with source-level separation across training, validation, and test sets. Each sample consists of a variable-length pre-incident clip, with its duration determined by the observable behavioral context preceding the incident. The inci- dent itself is excluded from all input clips. We evaluate three complementary information sources: facial-region appearance, temporally aligned audio, and body-motion features derived from tracked keypoints. Controlled ablations are performed with Swin-Tiny, ViT-Tiny, and DeiT-Tiny to measure the contribution of each modality under the same split. Results show that combining all modalities is more effective than using any other combination alone. The best configuration, Deit-Tiny with audio, facial appearance, and motion, achieves 91.21% accuracy, 88.96% balanced accuracy, 93.65% F1-score, and 96.38% ROC-AUC on the held-out test set. These results suggest that complementary appearance, acoustic, and kinematic cues provide useful evidence for recognizing elevated pre-incident risk.