SatOV improves remote sensing image segmentation using spatial restoration
SatOV: Restoring Spatial Priors for Training-Free Open-Vocabulary Segmentation in Remote Sensing Imagery
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Segmenting images from satellites is hard because these images need detailed understanding of spatial information that regular image models miss. The authors created SatOV, a method that restores important spatial details lost in typical models without needing training. They do this by combining attention from different layers and using the original high-resolution image to guide finer details. Tested on several remote sensing datasets, SatOV performs better than existing training-free methods and competes well with top approaches.
What this means in practice
- •For geospatial analysts: Segment satellite images for mapping land use or urban planning without retraining models for each new task.
- •For environmental monitoring teams: Identify features like roads, buildings, or vegetation in new geographic areas efficiently from remote sensing images without extra training data.
Authors
Changhao Zhao, Linglin Zeng, Hai Liu
Abstract
Open-vocabulary semantic segmentation (OVS) of remote sensing imagery is a challenging pixel-level task requiring strong generalization and adaptation to the spatial characteristics of remote sensing data. Although existing vision-language foundation models perform well in general domains, their image-level classification design weakens the spatial priors needed for high-resolution remote sensing segmentation: structural spatial relations are degraded during deep feature transformation, and fine-grained spatial details are lost during downsampling. To address these complementary deficiencies, we propose SatOV, a training-free framework for open-vocabulary remote sensing segmentation that restores spatial priors at two stages of the representation pipeline. Specifically, Residual QQ Attention (ResQQ) extracts Query-Key self-attention from an intermediate CLIP layer and fuses it with final-layer Query-Query attention via a residual combination, restoring structural spatial priors suppressed by the final-layer representation. Spatially Modulated Upsampling (SatUp) uses the original high-resolution RGB image as spatial guidance, combining spatial feature modulation with guided cross-attention to reconstruct pixel-level textures and boundaries. Extensive experiments on DOTA, UDD, LoveDA, and Vaihingen show that SatOV consistently improves training-free OVS and achieves competitive quantitative and qualitative results against state-of-the-art methods. These results validate the effectiveness of restoring spatial priors at both the representation and spatial-resolution stages for remote sensing open-vocabulary segmentation.