Summary
Navigating drones in large warehouses to find and follow people using natural language commands is a challenging problem. The authors created a new virtual environment and dataset called WareFly-VLA, where drones receive detailed language descriptions to locate and track workers in cluttered warehouse scenes. They tested several existing models for controlling drones with language input and found current methods struggle to perform well, especially under difficult conditions and when generalizing to new scenarios. Their work provides a benchmark and resources to spur more research in language-guided drone navigation for warehouse automation.
What this means in practice
- •For warehouse automation teams: Use language-guided drones to locate and track workers in complex warehouse settings for inventory management and worker safety monitoring.$Commercial implications: Enables commercial drone-based solutions for human tracking and navigation in warehouses, improving operational efficiency and safety.
- •For security monitoring teams: Deploy drones that can follow and locate specific individuals in industrial environments using natural language descriptions, aiding security and surveillance.
Authors
Thinh D. Le, Son T. Nguyen, Duong Q. Nguyen, Dung D. Le, Ngo Anh Vien, H. Nguyen-Xuan
Abstract
Vision-Language-Action (VLA) models have achieved impressive results in robotic manipulation and ground-mobile navigation, yet language-conditioned control of unmanned aerial vehicles (UAVs) in smart warehouses remains largely unexplored, hindered by the lack of benchmarks that jointly provide continuous low-level flight actions, fine-grained natural-language target descriptions, and realistic industrial environments. This paper introduces WareFly-VLA, a photorealistic UAV VLA framework and dataset for language-guided human search, localization, and tracking in warehouse environments. It contains 507 human-teleoperated flight episodes and 8,504 high-resolution RGB transitions collected in NVIDIA Isaac Sim, each paired with a human-written appearance description of the target worker and a synchronized four-degree-of-freedom control command. Two aerial tasks are covered: target approach and person following, under occlusion, long-range search, altitude variation, and clutter. A unified benchmark of four open-source VLA architectures (SmolVLA, GR00T N1.7, pi_0 and OpenVLA) is established under a leakage-free episode-level protocol at two control rates. The results show that language-conditioned aerial control in warehouses is far from solved: performance drops substantially under strict generalization settings, continuous action modeling consistently outperforms discrete action tokenization, only the forward channel is reliably learnable from a single frame, and current foundation-model interfaces transfer poorly from ground and humanoid embodiments to aerial platforms. The synchronized video, language, action, pose, and difficulty annotations further support world-model research. The dataset, baselines, and evaluation protocol are released to support language-grounded aerial autonomy in smart warehouses.