Embedding drones and ground robots cooperate to find vehicles better
Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Finding specified vehicles in busy city environments is hard, especially when using both flying drones and ground robots together. The authors created a new benchmark and dataset to test how well vision-language AI models can help these two work as a team by sharing what they see from different angles. They also designed a smart method that lets the AI models focus on understanding scenes and making decisions, while a clear cooperation plan handles the coordination between the drone and robot. Their tests showed this method helps most models find targets faster and more successfully, especially in difficult cases.
Unmanned Aerial Vehicle (UAV)Unmanned Ground Vehicle (UGV)Vision-Language Models (VLMs)Embodied AIObject searchBenchmark datasetMulti-view perceptionCooperative roboticsScene understanding
Authors
Boao Yu, Zimo Chen, Junreng Rao, Yue Hu, Zhengqiu Zhu, Yong Zhao, Rusheng Ju
Abstract
Air-Ground Object Search (AGOS) in urban environments is a challenging embodied task, which requires an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) to jointly search for and verify a specified target vehicle from multi-view visual references. To study this underexplored problem, we introduce AGOS-Bench, the first dedicated benchmark for evaluating whether general-purpose Vision-Language Models (VLMs) can integrate aerial discoveries and ground-level verification through UAV-UGV cooperation. We further provide AGOS-Dataset as the companion resource of exemplary trajectories constructed by an automatic pipeline. It consists of 7.7k episodes for searching objects of diverse categories and attributes, spanning three difficulty levels. To address the AGOS task, we propose AGOS-Agent, a training-free and tool-augmented approach. The agentic method relieves VLMs from complex and dynamic coordination via a deliberate search-handoff-verify cooperation protocol, only demanding VLMs for scene understanding and decision-making. Extensive experiments on nine VLMs show that AGOS-Agent improves overall success rate for eight of the nine evaluated backbones while reducing decision steps for all nine. On the hard split, the SR and SPL of Gemini-3.6-Flash increase from 8.6% to 55.7% and from 7.6% to 44.0%, respectively.