Active agents improve object location by choosing viewpoints efficiently
Towards Active Cross-View Object Geo-Localization
Computer Vision and Pattern Recognition
Summary
Figuring out exactly where an object is by looking at it from just one fixed camera angle can be tough. The authors propose a way for a moving agent, like a drone or robot, to actively pick new viewpoints to get better information and decide when it has seen enough to know where the object is. They trained their system in three stages to learn how to use multiple views together, follow smart paths, and balance accuracy with the effort spent looking around. Their tests show this method can locate objects well with fewer views than before, even without retraining on new environments.
What this means in practice
- •For drone operators: Enable drones to decide where to look next to quickly and accurately find target objects in outdoor environments.
- •For autonomous vehicle teams: Improve self-driving cars’ ability to locate objects by actively changing viewpoints during navigation for better scene understanding.
Authors
Shunyu Yao, Xiaohan Zhang, Zhuoran Yang, Haoqi Lai, Qi Ming, Xiaoxi Hu, Hui-Liang Shen, Si-Yuan Cao
Abstract
Cross-view object geo-localization (CVOGL) typically assumes a fixed query image, overlooking the ability of mobile agents to actively acquire more informative observations. To address this limitation, we introduce Active Cross-View Object Geo-Localization (ActiveGeo), where an agent sequentially selects new viewpoints and determines when to stop, aiming to improve localization with minimal observations. We further propose ActiveMoPT, an ActiveGeo framework with three-stage training. First, Multi-View Prompt-Preserving Adaptation enables the model to aggregate multiple query views while reusing the initial prompt. Second, Trajectory-Guided Policy Initialization uses supervised agent trajectories to learn viewpoint selection and initial stopping behavior. Third, Cost-Aware Policy Refinement employs GRPO with a gain-cost reward to jointly optimize localization accuracy and observation efficiency. We also construct ActiveGeo-858, a zero-shot test set containing 858 scenes and 1,716 target annotations. Experiments show that ActiveMoPT achieves state-of-the-art performance on MoP-UAV using only 1.45 query views on average, and substantially outperforms previous CVOGL approaches under zero-shot evaluation on ActiveGeo-858.