Real-Time Video Anomaly Detection Using YOLO Pose Estimation and CLIP-Based Semantic Scoring
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors created a fast, two-step method to find unusual actions in videos. First, their system spots people and finds important body points all at once. Then it looks at each person’s image and checks how similar it is to descriptions of strange behaviors. This way, they avoid using slower, complicated steps other methods need. Their tests showed it runs quickly and accurately on several video datasets.
video anomaly detectionYOLO v11n-poseskeletal keypointsCLIP ViT-B/32cosine similarityoptical flowpose estimationAUROCreal-time processingvideo datasets
Authors
Vanodhya G. Warnasooriya, Amir Hajian, Watchara Ruangsang, Supavadee Aramvith
Abstract
We propose a lightweight two-stage framework for real-time video anomaly detection. The first stage employs YOLO v11n-pose to detect persons and extract seventeen skeletal keypoints in a single forward pass. The second stage encodes each cropped person region through CLIP ViT-B/32 and computes cosine similarity against predefined textual descriptions of anomalous behaviors. This architecture eliminates the need for optical flow, standalone pose estimators, and density-based scoring modules. Experiments on CUHK Avenue, ShanghaiTech Campus, and a custom indoor dataset collected at Chulalongkorn University demonstrate an end-to-end throughput of approximately 51 FPS on an NVIDIA Titan XP GPU, a 3.36x speedup over the multi-feature baseline, while maintaining frame-level AUROC values of 89.26%, 70.26%, and 84.13%, respectively.