Geometry helps segment underwater objects beyond known categories
GeoPrompt: Geometry-Conditioned Prompt Learning for Underwater Open-Vocabulary Instance Segmentation
Computer Vision and Pattern Recognition
Summary
Underwater images are hard to analyze because water changes how things look, messing up usual methods that recognize objects using their colors and shapes. The authors propose using 3D shape information, which stays reliable underwater, to better identify and separate objects, even those the system hasn't seen before. Their method combines this shape data with text descriptions to improve recognizing underwater creatures and items. Tests show their approach works better than earlier methods on a key underwater image dataset.
What this means in practice
- •For marine robotics engineers: Improves underwater object recognition systems for robots by using 3D shape cues to identify unknown marine species or debris.
- •For environmental monitoring teams: Enhances automated analysis of underwater video to track new or rare marine life by integrating geometry-based classification.
Authors
Shanshan Han, Yuxi Wang, Can Qin, Yujie Chen, Cong Pan, Haitao Li, Guoqiang Zhong
Abstract
Open-vocabulary instance segmentation (OVIS) provides a promising solution for marine perception by recognizing and segmenting categories beyond predefined label spaces. Existing OVIS methods usually rely on vision-language alignment, where visual region features are matched with textual category embeddings to recognize both seen and unseen classes. However, this alignment assumption becomes fragile in underwater environments. Color attenuation, scattering, turbidity, and low contrast distort visual appearance, causing region features to deviate from their corresponding semantic concepts. As a result, the similarity between degraded visual features and textual classifiers becomes unreliable, which particularly weakens the recognition of unseen underwater categories that cannot be corrected by category-specific supervision. To address these issues, we propose GeoPrompt, a geometry-conditioned prompt learning framework for underwater open-vocabulary instance segmentation. Our key insight is that geometric structure is more stable than visual appearance in degraded underwater scenes. Specifically, GeoPrompt introduces a geometry-guided visual adaptation module (GVAM) to extract depth-derived structural priors and inject them into multi-scale visual features. It further develops a geometry-conditioned prompt learning (GCPL) that generates geometry-aware textual classifiers from structural representations. These geometry-aware classifiers are integrated with fixed and image-conditioned textual classifiers to improve open-vocabulary semantic alignment. Experiments on the MARIS-OpenIS benchmark show that GeoPrompt achieves state-of-the-art performance and brings consistent improvements on underwater open-vocabulary instance segmentation.