Cross survey machine learning predicts missing astronomical images
COSMIO: A Benchmark for Cross-Survey Modality Imputation
Computer Vision and Pattern Recognition
Summary
Astronomical surveys look at the sky in different ways, but any one object is only seen by some of these surveys, leaving gaps in information. The authors created COSMIO, a benchmark to test how well computer programs can fill in missing views of space objects across different surveys. They found that current methods struggle to predict rare cosmic events, like strong gravitational lenses, when these events only appear in some surveys. This makes it harder to detect such rare but important objects in new surveys. COSMIO helps measure and improve these prediction challenges systematically.
What this means in practice
- •For astronomy data teams: Evaluate and improve data completion algorithms to reconstruct missing images across multiple astronomical surveys for better object analysis.
- •For satellite imaging engineers: Develop methods to predict unobserved imaging modalities from different satellites to improve multi-sensor Earth observation data integration.
Authors
Dichang Zhang, Yixuan Shao, Jiaqi Deng, Jiali Cui, Haotian Yin, Yuanpeng Liu, Xiang Gao, Zhiqiang Lao, Heather Yu, Simon Birrer, Dimitris Samaras
Abstract
Large-scale astronomical surveys define a heterogeneous, partially observed multi-survey learning setting: each object is observed by only a subset of surveys, while survey observations differ in resolutions, wavelength coverage, noise characteristics, and footprints. This setting gives rise to a natural machine learning problem: given observations of an object in some surveys, how to infer its appearance in other surveys. We formalize this as the problem of cross-survey modality imputation and introduce COSMIO, a multi-survey benchmark of aligned observations obtained by forward-rendered shared astrophysical scenes across LSST, Euclid, and Roman under survey-specific instrument characteristics. The benchmark enables systematic evaluation of imputation across source-target survey combinations and missing-survey patterns. We evaluate a broad range of machine learning methods for imputation, which reveal a clear domain gap: methods developed for existing imputation settings transfer poorly to the survey setting. We identify rare-event imputation as a central challenge. This regime arises in the early stages of new surveys, when rare but scientifically valuable phenomena may be available only from prior surveys. We use strong gravitational lensing, where light from a background galaxy is gravitationally deflected by a foreground galaxy, as a representative rare-event setting. We evaluate model performance on zero-shot imputation of gravitational lenses and on downstream scientific tasks using the imputed lens observations. On our constructed evaluation set of galaxy-galaxy strong lenses, performance degrades under rare-event distribution shift, and this degradation further propagates to the downstream lens-detection task. Code: https://github.com/fw-ic/COSMIO.