FlyingDrones dataset improves drone motion estimation from images

FlyingDrones: A Dataset and Benchmark for Optical Flow Estimation from UAV motion

Computer Vision and Pattern Recognition

Summary

Tracking small drones in the sky using video is hard because they look tiny and blend into complex backgrounds. The authors created FlyingDrones, a new dataset with detailed motion information of drones in images, helping computer programs learn to better estimate drone movement. They tested many existing methods on this dataset and found which work best, especially at high image resolution. Their work also improves drone tracking on real videos after training on FlyingDrones data.

What this means in practice

  • For aerial surveillance teams: Improve detecting and tracking small drones in surveillance footage by using FlyingDrones data to better train motion estimation models.
  • For drone navigation developers: Use the FlyingDrones benchmark to fine-tune algorithms that localize and follow drones accurately in challenging, real-world environments.

Authors

Laurenz Ruzicka, Roman Pflugfelder, Daniel Cremers

Abstract

Optical flow is an important cue for drone analysis in aerial surveillance, airspace monitoring, and long-range target tracking, where motion remains informative when appearance is weak. Estimating the motion of small Unmanned Aerial Vehicles, however, is difficult because distant targets occupy few pixels, have low contrast, and move against complex dynamic backgrounds. Standard optical-flow benchmarks emphasize large, object-centric motions and therefore do not characterize distant aerial targets well. We introduce FlyingDrones, a generated benchmark with dense ground-truth optical flow and segmentation masks derived from physically based rendering, and define resolution-normalized metrics (EPE Norm) that normalize flow by image dimensions. We evaluate 16 state-of-the-art model families on a motion- and size-stratified subset of 999 flow pairs at three resolutions. Every family recovers drone motion better than the zero-flow reference, but the gap between families widens with resolution: at 1920x1080, global-matching, recurrent, transformer, and short-window temporal models separate clearly from pyramid, classical, self-supervised, lightweight, and diffusion models. Within-family checkpoint spread is comparable to or larger than between-family spread, so the training curriculum is as important as the architecture. Two complementary studies test the benchmark beyond synthetic accuracy: a ground-truth-free motion-alignment evaluation on 21 cameras of real UAV footage shows that the synthetic ranking carries a moderate, significant signal about real-world performance (Spearman \r{ho} = 0.63 over 41 checkpoints) while the exact top ranks differ, and a fine-tuning study shows that 800 FlyingDrones pairs reduce held-out drone-region error by up to 37% and improve drone localization on the real footage for all four architectures tested.