Foundation-Assisted Active Learning for Object Detection Annotation
Computer Vision and Pattern Recognition
Summary
The authors address the problem of high labeling costs in remote sensing object detection by creating a system that combines active learning with semi-automatic annotation. They use two sources of information—a reference localization system and a detector prediction—to better pick useful samples, especially early on when data is scarce. They also introduce methods to reduce noisy labeling and improve diversity in selected samples. Their approach reduces manual work and improves efficiency in dataset construction according to tests on several remote sensing datasets.
Authors
Jinchang Zhang, Arnold Zumbrun, Jing Lin, Guoyu Lu
Abstract
The annotation cost for remote sensing object detection is high, while existing active learning methods still face several challenges in object detection scenarios, including the coupling of localization and classification uncertainty, severe localization noise in the cold-start stage, and pseudo-diversity caused by high-recall candidate proposals. To address these issues, we propose a foundation-model-collaborative active learning and semi-automatic annotation framework for efficient construction of remote sensing object detection datasets. We build a dual-source mechanism consisting of a reference localization source (SA-source) based on UPN+SAM2 and a detector prediction source (OD-source), and further propose a Foundation-model-enhanced Dual-Source Uncertainty estimation to improve sample selection quality in the cold-start stage by jointly modeling localization consistency and classification confidence. Furthermore, we propose Object-Centric Diversity Sampling, which constructs object-level representations using DINOv2 features and SAM2 masks to improve sample coverage while suppressing pseudo-diversity. To address geometric noise in the semi-automatic annotation stage, we design Dual-Source Box Switching, which replaces noisy detector boxes with matched refined boxes from the SA-source, thereby reducing the manual burden of box refinement. Experiments on DIOR, HRSC2016, DOTAv2, and FAIR1M show that our method achieves superior or comparable results under most annotation budgets, with notably stronger cold-start sample efficiency in the low-budget regime.