Balanced prompt method improves binary image segmentation tests
Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
When computers try to divide images into two parts, they sometimes get stuck on the large part and miss the smaller one. This problem is because the usual way to adjust the computer’s guesses favors the bigger area too much. The authors found this issue and designed a new method that treats both parts fairly while updating only certain parts of the program. Their approach works better across different image types without changing the main image-understanding system.
What this means in practice
- •For medical imaging teams: Improve segmentation accuracy for medical scans where lesions are small compared to the background using balanced prompt adaptation.
- •For autonomous vehicle engineers: Enhance binary segmentation of road scenes with imbalanced objects like pedestrians and background for safer environment perception.
Authors
Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian, Xinxin Wang, Long Chen, Weiping Ding
Abstract
Entropy minimization is a standard objective for test-time adaptation (TTA), but it can fail in imbalanced binary segmentation. Unlike image classification, dense segmentation aggregates thousands of pixel predictions, allowing the larger predicted class to dominate the update, pull minority predictions toward itself, and produce a degenerate mask as predictions saturate and their entropy gradients vanish. We theoretically establish this collapse in a shared-shift model. This analysis motivates Balanced-Anchor Prompt Adaptation (BAPA), which combines two complementary modules. The Class-Balanced Anchors (CBA) module selects high-confidence anchors separately from each predicted class and gives foreground and background equal total loss weight, preventing the larger region from dominating the update. Dynamic Prompt Adaptation (DPA) refreshes these anchors after each prediction update and optimizes only text-side prompt residuals while keeping the vision-language encoders frozen. This prompt-only update refines the foreground-background decision boundary without altering the pretrained dense visual representation. Across experiments from four domains, BAPA achieves the highest mean Dice among the evaluated methods. Factorized ablations further validate the complementary roles of CBA and DPA, supporting balanced prompt adaptation as an effective alternative to entropy minimization for test-time binary segmentation.