Change captions help detect land changes in satellite images
From Change Captions to Change Detection: Semantic-Appearance Agreement Framework for Remote Sensing Change Detection
Computer Vision and Pattern Recognition
Summary
Detecting changes in satellite images usually needs detailed maps showing exactly where each change happened, which are hard to make. The authors use written descriptions called change captions that say what changed and where. They build a system that creates image pairs matching these captions and uses them to train a model to find where changes are, even without detailed maps. This approach works better than some weakly supervised methods on test datasets.
What this means in practice
- •For urban planners: Automate detection of land use changes from satellite images using caption-driven models without needing costly manual masks.
- •For environmental monitoring teams: Monitor environmental transformations by leveraging caption-based change detection to identify specific land-cover changes efficiently.
Authors
Yuan Qian, Jie Ma
Abstract
Remote sensing change detection (RSCD) is essential for monitoring land-cover changes and urban development. However, most methods demand pixel-level change masks, which are costly and time-consuming to annotate. Weakly supervised methods reduce this cost by using image-level change labels. Yet these labels indicate only whether a change occurs, leaving models to recover the location of the change and semantic meaning through additional and complex mechanisms. This missing information can be supplied directly by change captions, which describe what changes, what it becomes, and where it occurs. Therefore, we introduce change-caption-guided RSCD, using change captions as the sole task-specific supervision to learn change masks without manually annotated change masks. Our framework has two components: a caption-driven generation pipeline that produces bi-temporal remote sensing image pairs at scale with controlled changes matching each caption, and a change detector guided by the caption's transition semantics. The detector uses our Semantic-Appearance Agreement Framework (SAAF) to combine caption-grounded semantic responses with RGB differences for change localization, while text conditioning guides dense prediction. Experiments on our newly constructed Flair-RSGen dataset and WHU-CDC show that SAAF outperforms the closest reproduced limited-supervision baselines in macro-averaged IoU and F1 under the evaluated protocols. Code is publicly available at https://github.com/qianyuancs/SAAF.