A comprehensive and trustworthy benchmark of AI methods for change detection in Earth observation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors highlight that studying changes on Earth using satellite images is important but current methods are tested in different ways, making comparisons hard. They created a common testing setup that compares different types of deep learning models fairly on multiple datasets. Their results show simpler models often work better when considering both accuracy and speed, and using pre-trained models helps a lot without slowing things down. They shared all their data and code to make their work easy to use and verify.
Authors
Tadej Tomanič, Alice Baudhuin, Jan Sotošek, Jure Brence, Panče Panov, Nikola Simidjievski, Dragi Kocev
Abstract
Change detection in Earth observation (EO) is critical for monitoring land surface transformations, yet recent research in the field is constrained by inconsistent evaluation protocols and a narrow focus on predictive accuracy without regard for computational efficiency. To address this, we present a standardized, open-source benchmark for evaluating state-of-the-art (SOTA) deep learning methods for Earth observation change detection. We conduct a comprehensive analysis of ten representative model architectures, ranging from convolutional networks (CNNs) to vision transformers (ViTs), across ten heterogeneous change detection datasets. We rigorously evaluate these models with identical experimental protocols, comparing models trained from scratch against those utilizing pre-trained weights. Furthermore, we evaluate predictive performance alongside computational efficiency, including parameter counts and inference latency. Our findings reveal that well-optimized classical architectures, such as Siamese U-Nets, frequently outperform more complex contemporary models when computational efficiency is factored in, and that pre-training consistently provides a significant performance boost with no additional inference cost. To ensure complete transparency and reproducibility, all experimental resources, including standardized data splits, training scripts, training logs, and model checkpoints are publicly available and adhere to FAIR principles (Findable, Accessible, Interoperable, and Reusable).