Frozen hyperspherical features improve mask selection for image segmentation
Can Frozen Hyperspherical Features Guide the Selection of Pseudo Masks?
Computer Vision and Pattern Recognition
Summary
Choosing the best mask for an image is tricky because multiple possible masks exist and picking the wrong one leads to errors. The authors show that features from a frozen self-supervised model can help judge mask quality by looking at how a mask divides a special geometric space. They introduce SphereTrust, which scores masks quickly based on how well they separate features, cover image parts, and touch image edges. This method outperforms other approaches on varied images and also improves training of segmentation models.
What this means in practice
- •For computer vision engineers: Automatically rank candidate image masks to improve segmentation quality without extra annotations or models.
- •For machine learning practitioners: Use frozen backbone features to guide training data labeling and enhance segmentation model learning accuracy.
Authors
Xinge Guo, Fengyang Xiao, Dingming Zhang, Yuhan Chen, Rihan Zhang, Xingjian Li, Tianyang Wang, Chunming He, Sina Farsiu
Abstract
Foundation segmenters such as SAM return several plausible masks for an unlabeled image, and a student trained on the wrong one inherits its errors. Choosing among them means querying a second large model or fitting a quality head to annotated masks. We show that a candidate can be judged by what it does to a frozen self-supervised backbone's features. Normalized DINOv2 patch features lie on a hypersphere, and a candidate mask splits that sphere in two. Based on this reading, we introduce SphereTrust, which scores each candidate by three properties of the split, the angular contrast between the two sides, the coverage of the foreground's appearance modes, and contact with the image frame, one for each of three common ways a mask fails, and ranks a pool in 0.55 s per image from the frozen features alone. On eight SAM and SAM3 candidate pools spanning camouflaged, salient, and dichotomous segmentation and camouflage under low light, SphereTrust exceeds the strongest evaluated external baseline on six pools by 1.7 to 9.3 percentage points in mean selected Dice. These comparisons include published selection rules and explicitly labeled adaptations of DSS and UCOD-MKD. On the two prompted camouflage pools, its mean selected Dice is within 0.1 percentage points of the candidate-derived DSS adaptation, with a lower catastrophic-error rate. Which cue carries the signal depends on the candidate pool. The same sphere also supports training. The leading candidates enter as a candidate set with their scores as priors, prototypes reorder them, and a cross-fitted second round completes the labels, raising weighted F by 4.5, 2.3, and 5.5 points over fixed-label training on the three MLLM anchor pools, with students competitive with published unsupervised methods on nineteen test sets.