Probing Geospatial SSL Representations with Environmental Signals
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionMachine Learning
AI summaryⓘ
The authors studied how well self-supervised learning (SSL) models, which learn from satellite images without specific tasks, keep important environmental information like temperature and rainfall. They compared different SSL models and checked if these models reflected environmental variables that naturally affect satellite data, even though these variables were not used during training. They found that looking inside the models' representations gives extra insight beyond just testing on tasks, and that models better showing these environmental clues also did better on related tasks. They also shared a new dataset linking satellite images with environmental data to help future research.
Self-supervised learningSatellite imageryERA5 reanalysisEnvironmental variablesRepresentation learningSentinel-1Sentinel-2Spectral reflectanceRadar backscatterPANGAEA benchmark
Authors
Rohita Mocharla, Vishal M. Patel
Abstract
Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task. Most geospatial benchmarks evaluate representations solely through downstream tasks, providing limited insight into the information encoded within the representation itself. We ask a different question: do SSL representations of satellite imagery preserve statistical associations with environmental variables that co-vary with the imaging process? To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables, including temperature, precipitation, surface solar radiation, surface pressure, and volumetric soil water. These variables are physically related to the spectral reflectance and radar backscatter recorded by Sentinel-1 and Sentinel-2, making them meaningful evaluation targets despite not being used during SSL pretraining. We complement this probing analysis with intrinsic representation metrics to characterize representation geometry and investigate how these properties relate to downstream performance and the encoding of environmental signals. Using DINO, MAE, and MoCo models trained under identical conditions, we show that representation-level metrics distinguish models with similar downstream benchmark performance, providing complementary information beyond task-driven benchmarks. We further find that the linear accessibility of environmental signals is associated with performance on environmentally dependent tasks in the PANGAEA benchmark. Finally, we release ERA5 annotations co-located with the SSL4EO dataset to enable physically grounded representation evaluation for future geospatial foundation models.