Toward Active Object Detection for UAVs in the Wild: A Large-Scale Dataset, Benchmark and Method
2026-07-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionRobotics
AI summaryⓘ
The authors highlight challenges in detecting objects from drones, like parts being hidden or targets appearing very small. To address this, they created ATRNet-LUDO, a large dataset with many aerial images useful for training and testing detection methods that actively choose where to look. They tested current approaches and found they don’t generalize well to new scenarios, so they developed an improved method called AOD-JEPA to better understand drone views. Their work provides new tools and data to help researchers improve active object detection from drones.
Unmanned Aerial Vehicle (UAV)Object DetectionActive Object Detection (AOD)Deep Reinforcement Learning (DRL)Dataset BenchmarkPanoramic ImagingState Representation LearningJoint Embedding Predictive Architecture (JEPA)Active VisionGeneralization Gap
Authors
Tianpeng Liu, Xinhua Jiang, Li Liu, Qinmu Shen, Siwei Tang, Zhen Liu, Yongxiang Liu
Abstract
Object detection is a fundamental component in numerous Unmanned Aerial Vehicle (UAV) applications, yet it has long been plagued by hindrances like occlusion or target pixel scarcity. Active Object Detection (AOD) provides a novel paradigm to address these challenges via active vision, while UAV-based AOD research remains scarce due to the lack of high-quality datasets and benchmarks for algorithm development and evaluation. To fill this gap, this paper presents ATRNet-LUDO, the first large-scale real-world dataset for UAV-Ground Active Object Detection (UGAOD). It contains 121,000 multi-view panoramic multi-target aerial images and 1.21 million local single-target slices, covering 10 vehicle targets across 40 scenarios. It enables the construction of diverse training and testing environments for UAV agent interaction and active observation policy learning. Based on this dataset, we establish a comprehensive evaluation benchmark for AOD policy learning methods. Most existing AOD policies rely on Deep Reinforcement Learning (DRL) but suffer from poor generalization. Evaluations on our benchmark reveal a significant generalization gap between training and testing performance, highlighting an urgent need for solutions. To this end, we leverage the Joint Embedding Predictive Architecture (JEPA) to construct a world model that enhances state representation learning, and propose AOD-JEPA by incorporating AOD-specific prior knowledge. Extensive experiments validate its effectiveness and superiority. We hope ATRNet-LUDO and the benchmark will advance research in the UGAOD field. The dataset and code are soon available at https://github.com/Leo000ooo/LUDO_dataset.