Vision language models improve remote sensing with better data
Data-Driven Vision-Language Models for Remote Sensing: A Survey
Computer Vision and Pattern RecognitionMachine Learning
Summary
Remote sensing uses images from satellites or drones to understand the Earth, but teaching computers to interpret these images can be tricky. The authors show that instead of changing complex computer models, it helps more to focus on using better data specifically made for remote sensing combined with language descriptions. They reviewed existing datasets and shared practical advice on how to create and use these data collections to train smarter vision-language models. This helps pave the way for more general and versatile AI tools that understand Earth images across many tasks and places.
What this means in practice
- •For geospatial data engineers: Create and fine-tune vision-language models for more accurate interpretation of satellite images using tailored remote sensing datasets.
- •For environmental monitoring teams: Use improved vision-language models to better analyze environmental changes by leveraging diverse and high-quality remote sensing datasets.
A survey. It maps existing work.
Authors
Yue Zhou, Shujun Zhao, Xue Yang, Ruigang Li, Tianwen Zhang, Mengcheng Lan, Chaofeng Chen, Lingfei Ma, Hongjie He, Jonathan Li
Abstract
During the deep learning era, innovations in remote sensing (RS) vision models primarily focused on optimizing network architectures for specific tasks and conducting end-to-end training. However, with the advent of the Vision-Language Models (VLMs) era, constrained by substantial computational resource demands, the mainstream research paradigm has shifted towards adapting general-purpose VLMs using RS vertical domain data for fine-tuning, rather than modifying their core architectures. This transition marks a change in the innovation pathway from model-centric to data-centric. Despite the increasing importance of data, the academic community still lacks a systematic, data-centric survey to guide researchers in constructing high-quality RS vision-language datasets for various application scenarios and in conducting effective model fine-tuning and performance evaluation based on these datasets. To address this gap, this paper presents a systematic review of the development status of RS datasets tailored for VLMs. It further provides a comprehensive practical guide for RS researchers in the large model era, covering the entire pipeline from data construction to fine-tuning and evaluation. This paper aims to encourage the academic community to build more diverse and higher-quality vision-language datasets for specialized RS sub-fields. Ultimately, the goal is to converge these efforts into a comprehensive RS vision-language corpus that offers extensive task coverage and a wide geographical distribution, thereby laying a solid data foundation for advancing towards general artificial intelligence in RS.