Heterogeneous robot team improves target search with natural language
HEROIC: Heterogeneous Evidential Reasoning for Open-Vocabulary Identification and Cross-Robot Collaboration
Robotics
Summary
Finding people or objects in a tricky environment can be hard for robots because drones see from above but can't see details, and ground robots see well up close but move slowly. The authors created HEROIC, a system where flying and ground robots talk in natural language to decide who does what. This system lets drones adjust their flying height and switch roles when needed, helping the team work better together. Their tests show HEROIC locates targets more often and faster than other methods.
What this means in practice
- •For disaster response teams: Coordinate air and ground robots in disaster zones to improve search and rescue mission success and reduce time to find victims.
- •For urban security teams: Deploy aerial and ground robots communicating naturally to better locate and verify suspicious objects or persons in complex urban settings.
Authors
Mihir Chauhan, Aarav Jain, Addison Zucek, Manmeet Dang, Damon Conover, Aniket Bera
Abstract
Multi-agent heterogeneous air-ground robot teams are attractive for open world search, with applications for reconnaissance, urban search and rescue missions (USAR), disaster response and recovery, and hazardous environments. These two platforms have different failure modes: aerial robots cover ground quickly but cannot resolve small or occluded targets from altitude, while ground robots can identify objects-of-interest, such as people or hazardous objects, at close range but cover less area. Existing language-tasked teams either have roles fixed prior, or have a language model assign them from hand-written capability tags, so the team is unable to know when within a mission an asset is no longer useful. We present HEROIC, a decentralized heterogeneous multi-agent open-vocabulary search coordination framework that requires agents to communicate in natural language only. HEROIC's initial agent role assignment is derived from sensor properties and a scale law to determine whether targets can be detected with a high confidence. From the mission's natural language prompt alone, this law assigns aerial flight altitudes and sweep spacing. When this calculated height falls below the altitude for safe flight, aerial agents re-task themselves from searcher to aerial triage, escort, and route guide for ground agents. Both robots maintain an evidential belief over the search area (bearing rays for positive evidence, a log-odds posterior for negative evidence) and gate any arrival on close-range verification. In full-stack experiments, HEROIC reaches the target 84% of the time across all 6 scenes, compares to 35-54% for vision-language frontier baselines, frontier-based search, lawnmower, and random-walk running the same perception, all while being 2-4x sooner to arrive at the target.