UAV swarm agent turns simple commands into coordinated drone missions
UAV-SwarmAgent: Scalable Multi-UAV Reconnaissance and Task Allocation via Code Generation and Vision-Language Reasoning
Robotics
Summary
Controlling many drones at once for searching or inspecting is usually complicated. The authors present a system called UAV SwarmAgent that lets an operator give a simple natural-language command. The system then figures out which drones go where and what they should look at using AI models that understand both images and words. Simulations show it can find incidents reliably and that adding more drones speeds things up but with limits.
What this means in practice
- •For drone operators: Turn a written mission goal into coordinated flight routes for multiple drones automatically, reducing manual planning.
- •For emergency response teams: Deploy drone swarms that efficiently divide inspection tasks over an area with minimal operator input, improving incident detection.
Authors
Oleg Sautenkov, Yasheerah Yaqoot, Muhammad Ahsan Mustafa, Grigorii Kalinin, Chih-Yung Wen, Dzmitry Tsetserukou
Abstract
Supervising a multi-UAV reconnaissance mission usually means plotting waypoints and watching every drone. We present UAV SwarmAgent, a hierarchical multi-agent system in which an operator states the mission in one natural-language sentence and a ground-station agent writes and executes code that turns it into per-UAV routes. The agent grounds inspection targets in an aerial map with a vision-language model, splits them among N UAVs by balancing route length, and dispatches worker agents that verify incidents with a compact onboard decision model. In simulation over 4 incident types and 3 foundation models, the swarm located the incident in 244 of 255 missions (95.7%). A paired scaling study with one to 5 UAVs shows that flight time follows an Amdahl-type law. The second UAV delivers half of the attainable saving and the speedup is bounded at 1.52x. We discuss what this predictability, and the false alarms we observed, imply for an operator who supervises the swarm, and outline the user study needed to test it.