Terramesh-masks improves earth observation image segmentation with open vocabulary
TerraMesh-Masks: Open-Vocabulary Segmentation for Earth Observation
Computer Vision and Pattern Recognition
Summary
Segmenting Earth observation images into meaningful parts is hard because such images differ a lot from regular photos. The authors created TerraMesh-Masks, a huge dataset with millions of annotated segments for satellite images that link to real-world map data. They tested a segmentation tool on this dataset and got much better results than popular existing models. This work helps computers understand and identify many types of land and structures in satellite images without needing retraining.
What this means in practice
- •For geospatial data teams: Create more accurate and flexible land-use maps from satellite images without needing to train new models for each new category.
- •For urban planning teams: Automatically identify and segment hundreds of urban features from aerial images to support city development projects.
- •For environmental monitoring services: Deliver better automated segmentation of natural and man-made features in satellite data for environmental impact assessments.$Commercial implications: Makes possible commercial products that analyze satellite imagery for environmental risks and land changes using broad class definitions.
Authors
Benedikt Blumenstiel, Hugues Devimeux, Johannes Jakubik, Konrad Schindler
Abstract
Open-vocabulary segmentation (OVS) enables models to segment a variety of concepts specified in natural language, removing the need for fixed label sets and fine-tuning. While recent OVS approaches have shown strong performance on natural images, they transfer poorly to Earth observation (EO) data due to differing semantics, scales, sensing modality, and geographic context. To date, large-scale EO-specific datasets and benchmarks for OVS remain scarce. We, therefore, introduce TerraMesh-Masks, a large-scale open-vocabulary segmentation dataset for EO, comprising 16 million binary masks aligned with nine million multimodal TerraMesh samples and annotated using OpenStreetMap and Overture-derived semantics. In addition, we provide an expert-reviewed evaluation set of 523 samples covering over 200 classes to enable robust evaluation. To demonstrate the impact of TerraMesh-Masks, we build an open-vocabulary version of TerraMind. Our results demonstrate significantly improved performance over general-purpose state-of-the-art models reaching 35.18% IoU versus 16.29% for DINOv3.txt and 11.33% for SAM-3. We release the dataset publicly under a permissive license at https://huggingface.co/datasets/ibm-esa-geospatial/TerraMesh-Masks.