From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors worked on making drones better at following spoken or written instructions by looking around in 3D spaces. They found that drones struggled with recognizing important landmarks, remembering what they saw for a long time, and making good choices when stuck or confused. To fix this, the authors created a system that helps drones understand objects and their positions, remember important past views, and make smarter decisions by considering local and overall goals. Their method was tested and performed better than previous approaches on popular drone navigation tests.
UAVVision-Language NavigationEgocentric Visual ObservationsSemantic EnhancementTemporal AggregationLandmark RecognitionPolicy Optimization3D NavigationAerialVLNOpenFly
Authors
Zeyuan Ma, Jiaxin Chen, Di Huang
Abstract
UAV vision-language navigation (UAV-VLN) focuses on enabling an aerial agent to follow natural-language instructions in open 3D environments from egocentric visual observations. Current approaches suffer from three coupled issues: weak grounding of instruction-relevant landmarks in visual observations, insufficient exploitation of long-horizon history, and unstable decisions under local traps or repeated exploration. To address these issues, we propose a unified semantic-to-decision framework. First, we present an instruction-grounded semantic enhancement module that injects object-level semantics and relative spatial cues into the current observation state. Subsequently, we develop a relevance-aware dynamic temporal aggregation strategy that reweights the full history buffer while converting a few high-relevance frames into structured landmark prompts for the decoder. Finally, we devise a topology-aware decision method that combines local-optimum cognition with group-relative policy optimization under progress, goal, semantic, and path-compliance rewards. Experiments on the widely used AerialVLN and OpenFly benchmarks clearly demonstrate that our method achieves state-of-the-art performance.