Hypersam builds general model for analyzing earth images
HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing
Computer Vision and Pattern Recognition
Summary
Hyperspectral remote sensing collects detailed color information from the earth's surface, which helps identify materials and track changes. The authors created HyperSAM, a new computer model that can work with these complex images more effectively by combining real data and synthetic images. It uses another model called SAM3 to better understand objects in the images and smart techniques to handle imperfect labels. This approach lets HyperSAM perform well on various tasks like detecting changes, finding targets, and spotting anomalies in earth observation data.
What this means in practice
- •For earth observation teams: Process hyperspectral satellite data to classify materials and detect anomalies more accurately using a promptable foundation model.
- •For environmental monitoring companies: Map oil spills and other environmental hazards by applying a model trained on synthetic and real hyperspectral data for better detection.$Commercial implications: Enables commercial environmental services to offer higher quality oil spill detection from airborne hyperspectral images.
Authors
Li Pang, Xinqiao Wu, Jing Yao, Pedram Ghamisi, Jun Zhou, Zhengchao Chen, Deyu Meng, Xiangyong Cao
Abstract
Hyperspectral remote sensing provides dense spectral measurements that are indispensable for material-level Earth observation, yet the construction of a general-purpose hyperspectral foundation model remains difficult. Two bottlenecks are especially limiting. First, large hyperspectral corpora rarely provide high spatial resolution together with reliable dense annotations. Second, many hyperspectral models are still trained almost from scratch, so the geometric and interactive priors learned by modern vision foundation models are not fully reused. To alleviate these issues, we \highlight{present} \textbf{HyperSAM}, a promptable hyperspectral foundation model that couples a data-centric hyperspectral synthesis pipeline with a spectral adaptation architecture based on Segment Anything Model 3 (SAM3). On the data side, HyperSAM synthesizes full-spectrum hyperspectral cubes from high-resolution SpaceNet multispectral imagery through a physics-informed abundance-transfer generator, while SAM3-derived pseudo-masks provide object-centric supervision. On the model side, the latest implementation uses a frozen SAM3 RGB image branch, a trainable hyperspectral side encoder initialized from the RGB vision transformer (ViT), ControlNet-style zero-initialized feature injection, and a lightweight mixture-of-experts mask refiner. To enhance training robustness against noisy pseudo-labels, Cross-modal Sample Selection (CromSS)-style confidence selection is incorporated for noisy-label weighting. Extensive experiments show that HyperSAM obtains strong generalization on diverse hyperspectral tasks (e.g., classification, anomaly detection, change detection, target detection, and airborne oil-spill mapping) and that high-quality synthetic hyperspectral data can be more effective than simply scaling noisy hyperspectral supervision.