Parameter scale improves aerial video AI accuracy less than humans do

The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understanding

Computer Vision and Pattern Recognition

Summary

Aerial and remote sensing videos are usually tested using methods designed for regular web videos, but this paper argues that such tests miss important aspects of the task. The authors analyzed many previous studies and found that simply increasing AI model size only slightly improves accuracy on aerial videos, much less than the gap between AI and humans. They also showed that changing flight conditions like altitude or object size strongly affects performance, and current benchmarks do not fully capture the difficulty or human-level understanding needed. From these findings, the authors offer new principles and a reporting framework to better evaluate aerial video AI.

What this means in practice

  • For drone operations teams: Use the AVU-Card to better assess aerial video algorithms under varied flying conditions for mission planning.
  • For autonomous vehicle developers: Adopt the paper’s principles to design benchmarks that more accurately measure AI perception and action in aerial navigation tasks.

A position paper. It proposes an approach and reports no results.

Authors

Georgios Angelidis, Aggelos Psiris, Maria Vakalopoulou, Georgios Th. Papadopoulos, Giorgos Sfikas, Iraklis Varlamis

Abstract

Aerial and remote sensing video is increasingly evaluated like web video, by posing multiple-choice questions to multimodal large language models (MLLMs) over sampled frames and reporting accuracy. This paper argues that this recipe measures the wrong aspect, and supports the claim with a convergent meta-analysis of a broad set of published results, organized in four evidence tiers, in which a claim becomes a position only when collected evidence agrees. Parameter scale exhibits a median of +3.4 accuracy points per tenfold increase on aerial benchmarks, far below a human gap of at least 19 points, whereas acquisition conditions control perception, as 315 of 318 evaluations in 19 within-paper contrasts lose accuracy when conditions such as altitude, view, object size or time horizon become harder. No aerial benchmark in the corpus combines video input with per-task human and random baselines, competence collapses from answering to acting in closed-loop navigation, and the same MLLMs score higher on the aerial benchmark than on ground spatial ones, but lower on studies that test cross-view geometry. From these findings the current study derives five falsifiable principles and proposes the AVU-Card, a reporting card for aerial video benchmarks
View PDFOpen arXiv