Language guided image fusion improves satellite image interpretation accuracy
Bridging Low-Level Image Fusion and High-Level Semantic Understanding
Artificial IntelligenceInformation Retrieval
Summary
Combining different satellite images into one clearer picture usually focuses only on making the picture look good, not on helping computers understand what's in it. The authors introduce a method called LANCE that uses language descriptions as a bridge between making the image clearer and making sure the computer can still recognize important details. They check the consistency of these descriptions in several ways to improve both image quality and understanding without extra training on detailed labels. Tests on many satellite image sets show LANCE can make these images easier for machines to interpret.
What this means in practice
- •For remote sensing analysts: Improve image fusion workflows that produce satellite images easier to interpret automatically with semantic summaries.
- •For aerial imagery software developers: Build systems that generate fused satellite images with integrated text descriptions to support better downstream object detection.
Authors
Guangyuan Dong, Lingdong Shen, Xudong Zhang, Haodong Jing, Haoyu Wang, Zihao Li, Zonghao Ying, Ziyu Song, Shenghao Liu, Jiachen Luo, Hao Zhang
Abstract
Remote sensing image fusion and semantic interpretation are usually optimized separately, so reconstruction quality need not preserve downstream retrieval utility. We propose LANCE, which uses structured language as an explicit, indexable semantic state. A frozen vision-language model maps the input to a closed-vocabulary text record; a language-conditioned generator fuses the imagery; and a dense verifier with a differentiable continuous summarizer re-describes the output. Text-text consistency and anti-collapse contrastive text-mask losses with negative queries close the loop without dense annotations or VLM fine-tuning. A variational EM formulation yields a per-step contraction result under regularity assumptions, distinguishing semantic faithfulness from verifier overfitting. Across seven test sets on GF2, QuickBird, WorldView-III, and WorldView-II, LANCE improves reduced-scale reference metrics by up to 11.7% (median 2.1%) over the strongest learning baseline. Independent evaluation using a held-out VLM and a supervised aerial detector yields gains of +6.4 mAP and +0.072 open-vocabulary F1 over the closest competitor. Text perturbations, cross-verifier transfer, and pseudo-RGB ablations examine the mechanisms underlying these gains. The framework connects image fusion to verifiable semantic representations for multimodal retrieval and knowledge management.