PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors found that the popular Segment Anything Model (SAM) struggles when images have problems like noise or blur. To fix this, they created PGE-SAM, which uses user hints and previous guesses to focus on important parts while improving image features. They also made a special way to recover tiny details and only teach the model about relevant areas. Additionally, the authors made a new test set called DM-Seg to check how well methods work on damaged medical images. Their experiments show that PGE-SAM is better at handling messy images than previous methods, without making the model much bigger.
Segment Anything Model (SAM)image segmentationzero-shot generalizationfeature restorationprompt-guided learningmulti-scale featuresmedical imaginginteractive segmentationimage degradationbenchmark dataset
Authors
Tuan-Duc Nguyen, Anh-Tuan Mai, Duc-Trong Le
Abstract
Segment Anything Model (SAM) has revolutionized promptable image segmentation with strong zero-shot generalization. However, its performance degrades substantially under real-world imaging artifacts such as noise, blur, and compression. Existing methods restore features globally without focusing on segmentation-relevant regions and neglect SAM's iterative refinement mechanism, leading to suboptimal performance in interactive settings. We propose Prompt-Guided Feature Enhancement SAM (PGE-SAM), a framework that explicitly leverages user prompts and prior mask predictions to spatially guide the feature restoration process toward regions of interest through a Prompt Guidance Generator. To recover fine-grained details lost under degradation, we introduce Multi-Scale Features Interaction to incorporate low-level encoder features, along with a Foreground Reconstruction Loss that restricts feature-level supervision to the segmentation target. Furthermore, we present DM-Seg, a benchmark for interactive segmentation on degraded medical images, spanning multiple imaging modalities with both general and modality-specific degradations at varying severity levels. Extensive experiments demonstrate that PGE-SAM achieves SOTA robustness on both medical and natural image domains across multiple degradation levels, while maintaining generalization to clean images and adding less than one-fifth of the parameters of prior methods.