AI summaryⓘ
The authors developed a new way to teach computers to understand satellite images by focusing on the mix of land types within each image area, rather than treating each image as one single concept. They represent each area as a histogram showing the different land covers present and train their model to predict these compositions. This approach helps the model better recognize and compare complex scenes from satellites, improving tasks like finding similar images and classifying scenes. Their smaller model outperforms much larger existing models on several tasks, especially on a test requiring detailed recognition of mixed land covers. This shows that explicitly modeling land cover mixtures helps the model learn more effectively.
Geospatial foundation modelsPretrainingLand-cover compositionHistogramEarth Mover's DistanceZero-shot image retrievalScene classificationSegmentationObject detectionmAP@10
Authors
Aryan Kashyap Naveen, Abhishek Srinivas, Pranav Moothedath, Shrutilipi Bhattacharjee
Abstract
Geospatial foundation models have emerged as state-of-the-art methods for downstream Earth observation tasks. However, existing pretraining methodologies process imagery through a single-concept lens, failing to capture the highly compositional nature of complex satellite scenes. We propose a composition-aware pretraining framework that explicitly encodes fractional land-cover mixtures. Each satellite image cell is mapped to a histogram representing its fractional land-cover distribution, which we term the "composition target". These targets serve as the primary prediction objective and are distilled into the backbone using Earth Mover's Distance. Experimental evaluation shows that composition-aware pretraining yields substantial gains on region-level understanding tasks requiring semantic similarity judgment, including zero-shot image retrieval and scene classification, while remaining competitive on tasks requiring fine-grained spatial precision, such as segmentation and object detection. With a 36.8M-parameter backbone, our framework outperforms SatMAE and Prithvi-EO-2.0, which contain 303M and 600M parameters, respectively, in most retrieval and scene classification settings. On the fine-grained ForestNet-12 dataset, a rigorous testbed for compositional discrimination, our method boosts baseline mAP@10 from 0.279 to 0.434, a 55.6% relative improvement, providing direct evidence for the effectiveness of explicit composition modeling. The code implementation can be found at https://github.com/05kashyap/GFM_Composition_Pretraining