Vision conditioned model improves camouflaged object detection with text prompts
ViCo-SAM3: Vision-Conditioned Alignment for Open-Vocabulary Camouflaged Object Segmentation
Computer Vision and Pattern Recognition
Summary
Picking out camouflaged objects from pictures is hard, especially when you want to find things that the system hasn’t seen before, based only on text descriptions. The authors noticed that existing methods struggle because the connection between overall text meanings and tiny visual details is weak. They made a new method called ViCo-SAM3 that adjusts the text hints using clues from the image itself, so the text can better match what’s actually in the picture. This makes the system much better at segmenting camouflaged objects it hasn’t encountered before.
What this means in practice
- •For security monitoring teams: Improve detection of hidden or camouflaged objects using flexible text queries in surveillance footage.
- •For wildlife researchers: Automatically segment and study camouflaged animals in natural habitats guided by descriptive text without prior species-specific training.
Authors
Qiangqiang Zhou, Wenjun Tang, Yong Chen, Dandan Zhu, Jiawei Xu
Abstract
Open-vocabulary camouflaged object segmentation (OVCOS) aims to segment unseen camouflaged objects under text guidance. We observe that SAM3 still suffers from a pronounced semantic gap between global textual semantics and fine-grained pixel-level visual cues in OVCOS. Meanwhile, fully fine-tuning the text encoder introduces heavy parameter overhead and risks overfitting to training categories, which compromises open-vocabulary representation flexibility. To address these issues, we propose ViCo-SAM3, a Vision-Conditioned alignment framework designed for OVCOS. Specifically, we introduce vision-conditioned (ViCo) module, which dynamically modulates text embeddings with global visual context, enabling textual representations to adapt to the current image content and thereby effectively bridging the semantic gap between vision and text. Building on this, we further design a vision-conditioned cross-modal binding (ViCoBind) module to enhance cross-modal interaction and semantic alignment between visual and textual representations. Without bells and whistles, ViCo-SAM3 achieves state-of-the-art performance on the OVCamo benchmark and demonstrates strong generalization.