Label guided method improves 3D CNN video action recognition
Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Video action recognition models often use simpler versions of image analysis techniques that ignore the timing in videos, which can reduce accuracy. The authors introduced a way to help smaller, faster video models learn better from bigger ones by using labels to guide the learning process at both individual video and category group levels. This approach helps the small models understand timing and category relationships better, which improves their accuracy. They tested this on standard video action datasets and found their method competitive with existing techniques.
What this means in practice
- •For mobile app developers: Improve lightweight video action recognition models on mobile devices by guiding their learning with labeled data and relational category information for better accuracy.
- •For video analytics teams: Enhance compressed 3D-CNN models for real-time action recognition in surveillance or sports analysis through label-guided feature learning that captures temporal and categorical information.
Authors
Yanjiang Shi, Peng Zhao, Nan Qi, Guiqin Wang
Abstract
As a key model compression technique, knowledge distillation aims to transfer knowledge from a high-capacity teacher model to a lightweight student model for enhancing the latter's performance. In this work, we reviewed the feature knowledge distillation for 3D-CNNs and observed that most feature distillation methods in video analysis are simple adaptations of those used in image analysis, often neglecting the differences of video features in the temporal dimension. To address this issue, we proposed Label-Guided Knowledge Distillation (LGKD) to guide the distillation of student model features using ground truth labels. Our method entails two components: sample-wise distillation and class-wise distillation, enabling the student model to learn feature representation of the teacher model at two levels. Sample-wise distillation utilizes label information and the teacher's probability distribution to guide the learning of features that significantly impact temporal accuracy while mitigating noise. Meanwhile, class-wise feature distillation employs a prototype network to further capture the relational knowledge among samples within the same category, enhancing the student's ability to learn higher-dimensional semantic information and improving model generalization. To demonstrate the effectiveness and superiority of our method, we conducted comprehensive experiments on two benchmark action recognition datasets, UCF101 and HMDB51, achieving competitive results.