RSLiveBench UAV benchmark tracks aerial spatial intelligence progress
RSLiveBench-UAV: A Continually Evolving Vision-Language Benchmark for Aerial Spatial Intelligence
Computer Vision and Pattern Recognition
Summary
Understanding objects and their locations from drone images is hard because the views and scales change a lot. The paper introduces RSLiveBench-UAV, a benchmark that keeps updating with new real drone pictures and questions about what’s seen. The benchmark links the same vehicles seen from different views and uses careful question and answer checks without guessing. They tested many AI models on various tasks like 2D and 3D perception and found that recognizing specific vehicles is still tough. This tool helps keep checking how well AI can understand the world from above.
What this means in practice
- •For drone software developers: Evaluate and improve AI models for interpreting spatial data from drones in changing environments and views.
- •For autonomous vehicle engineers: Test object recognition and spatial understanding systems using updatable, viewpoint-varied aerial data to improve vehicle navigation.
- •For geospatial data providers: Enhance aerial image analysis tools with a benchmark that tracks performance on repeated scenes and evolving data.$Commercial implications: Supports development of commercial aerial image analytics products by providing continuous evaluation on authentic UAV data.
Authors
Yue Zhou, Jiawei Liu, Jue Chen, Ziyang Gong, Haojia Wei, Jingxiang Li, Xue Jiang, Zhihang Zhong, Pai Peng, Xue Yang, Yansheng Li, Jonathan Li
Abstract
Aerial spatial intelligence (ASI) requires models to understand objects and spatial relationships across changing viewpoints and scales. Evaluating these abilities as models evolve calls for fresh physical observations, yet producing new aerial spatial questions requires repeatedly establishing entity identities, geometric references, and cross-view correspondences. We introduce \textbf{RSLiveBench-UAV}, a continually evolving vision-language benchmark driven by recurring acquisition of real UAV imagery. Its update protocol admits images captured after the preceding release and delays answer disclosure. An entity-centered production workflow uses geometry-assisted association to link repeated observations of physical vehicles, reuse human-verified attributes, and combine them with view-specific annotations and spatial references for traceable question generation. All tasks use deterministic scoring without model judges. We evaluate 30 general-purpose, remote-sensing, and embodied models on 35,491 question records from ten scenes, covering 17 tasks across 2D perception, 3D perception, cross-view perception, and reasoning. All 18 tested embodied models outperform the three selected remote-sensing comparators in Overall, although the strongest embodied model scores 43.29 versus 59.88 for GPT-6-Astra. Fine-grained vehicle recognition remains challenging, and performance varies across heights and camera pitches. Retrospective scene-group comparisons and rolling-window replay characterize score-pattern consistency under scene renewal. RSLiveBench-UAV connects fresh physical observations, entity-centered answer production, and systematic model evaluation to support continual assessment of ASI. Benchmark will release at https://rslivebench.space