LightVLN enables efficient drone navigation with compact memory use
LightVLN: Efficient Aerial Vision-and-Language Navigation with Compact Memory and History-Guided Local Aggregation
Computer Vision and Pattern RecognitionRobotics
Summary
Flying drones to follow spoken instructions in complex 3D spaces often uses costly computing that is hard to run on the drone itself. The authors introduce LightVLN, a lightweight system that shrinks both memory use and computation by compressing past images and summarizing current views with fewer details relevant to navigation. They tested LightVLN on public benchmarks where it outperformed larger systems and showed it can run fast enough on real drone hardware. This work makes it easier for drones to navigate using language instructions without heavy onboard computers.
What this means in practice
- •For drone navigation engineers: Run efficient vision-and-language navigation models on drones with limited onboard computing using compact memory and token aggregation techniques.
- •For robotics system integrators: Deploy navigation policies that balance success rates and computing speed on aerial robots in complex 3D environments using lightweight language and vision models.
Authors
Yiming Zhao, Tianshun Li, Jingle He, Ruonan Chai, Xinhu Zheng
Abstract
Aerial vision-and-language navigation (VLN) enables unmanned aerial vehicles to execute long-horizon natural-language instructions from visual observations in complex three-dimensional environments. However, recent aerial VLN models often rely on large-scale vision-language backbones and dense visual histories, imposing substantial computation and memory costs that hinder onboard deployment. We propose LightVLN, a lightweight history-aware aerial VLN framework that combines a compact 0.5B language backbone with compact representations of both historical and current observations. LightVLN compresses each historical frame into a single token using visual features already computed by the policy. It further introduces history- and instruction-conditioned local aggregation to reduce the current observation from 256 to 32 visual tokens while preserving navigation-relevant spatial information. With up to 16 historical frames, the policy uses at most 48 observation-derived tokens. On the public OpenFly dataset, LightVLN achieves 50.93% Test-Seen and 36.14% Test-Unseen success rates (SR), outperforming the evaluated 7B language-backbone baselines on most reported metrics. It also achieves 25.83% SR on AerialVLN-S Val-Seen. In a reconstructed unseen campus, we deploy LightVLN on a DJI M350 RTK with an external Jetson Orin NX 16 GB for closed-loop onboard-compute real-to-sim hardware-in-the-loop (HIL) evaluation, achieving 14.61 Hz model inference and 11.13 Hz end-to-end decision updates. These results demonstrate the effectiveness and efficiency of LightVLN for aerial navigation.