OpenEarth-OVS44 dataset boosts remote sensing land-cover segmentation
OpenEarth-OVS44: A Multimodal Remote Sensing Dataset for Open-Vocabulary Segmentation
Computer Vision and Pattern Recognition
Summary
Understanding land cover from satellite images is hard because there are many types of surfaces and things to recognize, and it takes a lot of work to label them all. The authors created a large and detailed dataset called OpenEarth-OVS44 with many expert-labeled images from different countries. This dataset helps computer programs recognize land types even when they haven’t seen them before and under different conditions like weather changes. Their tests show that training on this data improves how well these programs can identify specific land covers.
What this means in practice
- •For urban planning teams: Improve detailed land-use maps to support functional zoning using fine-grained recognition from satellite images.
- •For environmental monitoring groups: Better estimate carbon storage and change by identifying diverse land covers accurately even under cloudy or varied weather conditions.
Authors
Yimin Wei, Hongruixuan Chen, Hao Zheng, Hao Liu, Jian Song, Yuting Zhu, Junshi Xia, Naoto Yokoya
Abstract
In remote sensing, open-vocabulary segmentation is particularly important for flexible and scalable land-cover understanding across diverse regions, as it improves generalization to novel categories without requiring exhaustive pixel-level annotations. Despite its great promise, progress in this area remains largely constrained by existing datasets with limited category coverage and overly coarse taxonomies, and therefore struggles to support fine-grained semantic understanding required in real-world applications such as urban functional zoning, carbon accounting, and post-disaster assessment. To address these challenges, we present OpenEarth-OVS44, a multimodal benchmark with 290,587 expert-verified segments from 2,832 RGB images and 1,341 strictly registered RGB-SAR pairs at sub-meter spatial resolution, covering 40 cities across France, Japan, and the United States under a unified 44-class taxonomy organized into 9 base categories. Three evaluation tracks are provided: an RGB-only fine-grained benchmark, a multimodal track with diverse synthetic cloudy conditions for weather-robust evaluation, and a held-out cross-region track. Extensive experiments on representative open-vocabulary segmentation models demonstrate that training on OpenEarth-OVS44 substantially improves fine-grained segmentation performance over the strongest training-free baseline, with improvements of up to 7.1 mIoU points on unseen categories and 27.6 mIoU points on seen categories. The dataset and code will be released.