Video anomaly detection improves with routing modulation and mixture-of-experts
RoMod: Temporal Routing Modulation via Mixture-of-Experts for Video Anomaly Detection
Computer Vision and Pattern Recognition
Summary
Detecting unusual events in videos is hard because these events are rare and varied. The authors studied special neural network parts called mixture-of-experts to see which parts react to anomalies. They found that a small group of network experts focuses on different types of anomalies and their signals help identify unusual events better. Using this insight, they built a system called RoMod that detects video anomalies efficiently with less training data and faster speeds.
What this means in practice
- •For security system developers: Deploy faster video anomaly detection systems that need minimal training data and can identify different anomaly types dynamically.
- •For video surveillance operators: Enhance monitoring by integrating routing signals to improve detection of onset, persistence, and termination of unusual activities in video feeds.
Authors
Chao Huang, Pengfei Wei, Benfeng Wang, Chengliang Liu, Wei Wang, Li Shen, Wenqi Ren, Xiaochun Cao
Abstract
Intermediate-layer features from multimodal large language models have shown strong potential for video anomaly detection (VAD), yet the origin of their discriminative power remains unclear. We study this question using sparse mixture-of-experts (MoE) models, whose explicit expert structure and sparse activation make their internal computation easier to inspect. With a fully frozen backbone and no additional training, we find that anomaly-related evidence is concentrated in a small set of experts. These experts recur across layers, spontaneously specialize in different anomaly types, and together form a dynamic routing subnetwork. We further show that the output channels most strongly influenced by these experts are also the hidden dimensions that contain the most anomaly-relevant information. Routing statistics can therefore serve as an internal anomaly cue that complements semantic features.Based on these findings, we propose RoMod, an efficient VAD framework trained with only \(5\%\) of weakly labeled videos. RoMod includes a Routing-Modulated Fusion module, RoMF, and a Routing-aware Temporal Network, RoTN. RoMF uses routing signals to adaptively recalibrate hidden semantic channels. Its design also prevents the routing branch from bypassing semantic features and making predictions on its own. RoTN captures the temporal evolution of anomalies from onset to persistence and termination. Experiments on three benchmarks show that RoMod achieves state-of-the-art performance while running substantially faster than dense backbones of comparable size.