Air and ground drones cooperate to find objects in cities
Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Finding objects like vehicles using drones is hard because it needs both flying and ground robots to work together well. The authors created a new test and data to see if smart vision-language AI models can help drones share information to find targets better. They also made a system that lets these AIs guide the drones in searching and checking objects without complicated commands. Their tests showed that their system helps most AI models find things more reliably and faster in difficult city scenes.
What this means in practice
- •For drone system developers: Design cooperative aerial and ground drone systems that efficiently share discoveries to locate vehicles in complex urban environments.
- •For urban security teams: Implement drone teams combining air and ground views to quickly find and verify vehicles of interest in city settings.$Commercial implications: Enables commercial deployment of coordinated drone security patrols with enhanced search accuracy using vision-language AI.
Authors
Boao Yu, Zimo Chen, Junreng Rao, Yue Hu, Zhengqiu Zhu, Yong Zhao, Rusheng Ju
Abstract
Air-Ground Object Search (AGOS) in urban environments is a challenging embodied task, which requires an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) to jointly search for and verify a specified target vehicle from multi-view visual references. To study this underexplored problem, we introduce AGOS-Bench, the first dedicated benchmark for evaluating whether general-purpose Vision-Language Models (VLMs) can integrate aerial discoveries and ground-level verification through UAV-UGV cooperation. We further provide AGOS-Dataset as the companion resource of exemplary trajectories constructed by an automatic pipeline. It consists of 7.7k episodes for searching objects of diverse categories and attributes, spanning three difficulty levels. To address the AGOS task, we propose AGOS-Agent, a training-free and tool-augmented approach. The agentic method relieves VLMs from complex and dynamic coordination via a deliberate search-handoff-verify cooperation protocol, only demanding VLMs for scene understanding and decision-making. Extensive experiments on nine VLMs show that AGOS-Agent improves overall success rate for eight of the nine evaluated backbones while reducing decision steps for all nine. On the hard split, the SR and SPL of Gemini-3.6-Flash increase from 8.6% to 55.7% and from 7.6% to 44.0%, respectively.