Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation
2026-06-15 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem that medical images often have different expert annotations, which many methods ignore by assuming just one correct label. They introduce a new method using attention to adjust prototype representations for each expert's unique style without changing the main feature extractor. This allows the system to better reflect individual expert opinions while maintaining overall consistency. Their experiments show this approach improves accuracy on datasets with multiple expert annotations.
few-shot segmentationmedical imagingmulti-rater annotationsprototype learningattention mechanismssemantic consistencyfeature extractorannotation variability
Authors
Truong Vu, Minh Khoi Ho, Yutong Xie
Abstract
Few-shot medical image segmentation methods typically assume a single ground-truth annotation, overlooking systematic variability across expert raters commonly observed in clinical datasets. We propose an attention-based prototype calibration framework for few-shot multi-rater segmentation that models rater-specific deviations from a consensus representation in prototype space. A lightweight yet principled attention operator directly refines rater prototypes without modifying the backbone feature extractor, making the approach fully compatible with existing prototype-based few-shot segmentation methods. This design preserves semantic consistency while enabling personalized segmentation outputs with minimal computational overhead. Experiments on multi-rater medical imaging datasets demonstrate consistent improvements over baseline prototype approaches, highlighting the effectiveness of structured prototype calibration for modeling annotation variability. Our code is available at https://github.com/truong2710-cyber/JAPC.