Summary
Biomedical vision-language models can recognize images without training and perform well, but they may miss small visual differences important for certain medical tasks. The authors created a method called Prompt-Anchored Residual Adaptation (PARA) that combines a fixed baseline prediction with a new visual prediction learned from a few labeled examples to improve accuracy. PARA adjusts predictions in a way that is less sensitive to which examples are chosen, making results more reliable. They also introduced a new way to evaluate performance by testing with many different labeled example sets. Their method performs better than previous approaches in recognizing medical images with few examples.
What this means in practice
- •For medical imaging teams: Improve automated classification of biomedical images when only a few labeled examples per condition are available.
- •For digital pathology software developers: Develop enhanced tools to distinguish subtle visual differences in medical images by incorporating prompt-anchored adaptation techniques.
Authors
Jingxuan Kang, Qianying Yue, Che Liu, Chen Qin
Abstract
Pretrained biomedical vision-language models achieve strong zero-shot performance in biomedical image classification. However, downstream biomedical classification often depends on subtle visual differences between classes that may not be fully captured by pretrained representations. Few-shot adaptation addresses this mismatch by optimizing a task-specific predictor on a small labeled support set. Because the selected examples capture only part of the visual variation within the target classes, the adapted predictions can depend strongly on their composition. We propose Prompt-Anchored Residual Adaptation (PARA), which retains the frozen prompt prediction as a support-invariant semantic anchor and incorporates a visual prediction learned from the support set through an anchor-relative residual. The residual step is computed in a closed form from frozen support embeddings using anchor discrepancy and support agreement. Support-set dependence also limits evaluation: comparisons are fair within a shared draw but remain conditional on its composition. To obtain more reliable comparisons, we introduce a repeated-support protocol that separates support-selection variation from optimization randomness and reports both average and worst-20% performance. PARA achieves state-of-the-art performance in both few-shot classification and base-to-novel generalization.