Federated screening reduces video transmission for anomaly detection
Federated Binary Gating with Server-Side Vision-Language Inference for Surveillance Anomaly Classification
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Detecting unusual events in surveillance videos is hard when camera data can’t be shared in full due to privacy. The authors developed a two-step system where simple local models first flag suspicious clips, sending only those to a powerful server model to identify broad anomaly types. This approach works better than trying to classify all anomaly types locally in a federated way, especially for rare events, while cutting down the amount of video sent over the network. They show that tuning when videos are sent balances better detection accuracy with less data transfer.
What this means in practice
- •For security system integrators: Deploy privacy-preserving surveillance by locally screening videos to forward only suspicious footage to server models, reducing bandwidth without losing detection accuracy.
- •For cloud service operators: Manage server-side processing resources efficiently by using federated binary gating to limit the volume of high-cost video data analyzed for anomaly types.
Tested on one dataset.
Authors
Côme-Alexis Puech, Sébastien Thuau, Amira Gran, Arthur Mennessier, Siba Haidar, Rachid Chelouah
Abstract
Privacy-sensitive surveillance systems could benefit from large vision-language models (VLMs), but such models typically require centralized access to raw video. In federated learning settings, this challenge is amplified by non-independent and identically distributed (non-IID) client data, which can make direct multiclass anomaly classification unstable, especially for rare categories. We propose a hybrid two-stage architecture that combines a federated binary convolutional neural network (CNN) gate with server-side zero-shot VLM inference. The lightweight LiteCNN3D gate performs local anomaly screening and forwards only flagged videos to Qwen3-VL-8B, which assigns them to four anomaly metaclasses. We evaluate this design on UCF-Crime grouped into five coarse metaclasses and implement the federated stage in a real three-node heterogeneous deployment. In the studied setting, direct federated multiclass training collapses, whereas the proposed decomposition yields a better trade-off between classification quality and raw-video transmission. With fixed-threshold routing, the federated hybrid pipeline preserves nearly the same macro-averaged F1 score (F1-macro) as its centralized CNN+VLM counterpart while reducing the fraction of transmitted videos to 51.4%, although with a lower proxy macro receiver operating characteristic area under the curve (ROC AUC) than the centralized hybrid system. A complementary sensitivity-oriented routing operating point increases macro ROC AUC from 0.673 to 0.692 and reduces the false negative rate from 29.3% to 22.9%, but decreases F1-macro from 0.503 to 0.485 while increasing transmission from 51.4% to 57.9%. These results suggest that federation is better suited to coarse local screening, while routing rules can be adjusted to trade server-side VLM usage for higher anomaly sensitivity.