Cross-scale knowledge distillation improves tiny object detection speed
CSCWD: Cross-Scale Channel-wise Knowledge Distillation for Lightweight Tiny Object Detection on Edge Devices
Computer Vision and Pattern Recognition
Summary
Detecting tiny objects in aerial images is hard because they have little detail and lightweight detectors lose important resolution. The authors developed a training method called CSCWD that teaches a small detection model using a bigger model’s detailed knowledge but keeps the small model fast. This method boosts detection accuracy and recall without making the small model slower or bigger at run-time. They tested it on drone imagery and security datasets, showing better detection of small targets on low-power devices like Raspberry Pi.
What this means in practice
- •For drone operators: Improve real-time detection of tiny objects in drone surveillance using a lightweight model with enhanced accuracy and low latency.
- •For security system engineers: Enhance detection of small aerial threats on embedded hardware without increasing processing time or memory use.
Authors
Amir Zamani, Zeinab Ghasemi-Naraghi
Abstract
Real-time tiny object detection in aerial imagery is constrained by the weak spatial evidence of very small objects and the loss of high-resolution detail in lightweight detectors. This study presents Cross-Scale Channel-wise Knowledge Distillation (CSCWD), a training-time framework that transfers high-resolution spatial representations from a YOLO11m-P2 teacher to a compact YOLO11n student without altering the student's inference architecture. Unlike conventional same-scale feature distillation, CSCWD transfers supervision from teacher P2 to student P3 after feature alignment while retaining same-scale distillation at deeper pyramid levels. Under the unified seven-sequence Drone-vs-Bird validation protocol, YOLO11n-CSCWD achieves 50.17% mean average precision at an intersection-over-union threshold of 0.5 (mAP@0.5) and 59.73% recall, improving the matched CA-YOLO11n baseline by 2.92 percentage points in mAP@0.5 and 3.55 points in recall. Cross-scale alignment further increases mAP@0.5 by 2.09 points over the corresponding same-scale channel-wise distillation configuration. In zero-shot evaluation on DUT-Anti-UAV, mAP@0.5 increases from 48.29% to 50.06% without target-domain fine-tuning. This domain was included because its challenging small targets make low-latency, computationally efficient detection particularly relevant. On Raspberry Pi 5 using NCNN-FP16 at 640x640 resolution, the 2.58-million-parameter student achieves 50.32% mAP@0.5 at 82.32 ms mean wall-clock latency, or 12.15 frames per second, while retaining essentially the same runtime and memory requirements as the matched baseline. The results support cross-scale distillation for improving tiny-target detection without increasing inference-time model complexity.