UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging
2026-07-23 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of teaching one type of medical image analysis system using data from another type when those data sets are not paired together. They introduce UnDA, a method that aligns important features across different data types without needing them to be matched, using a new alignment and noise-reducing technique called Uncertainty-Weighted Optimal Transport. Their approach keeps track of class prototypes to maintain clear distinctions and improves accuracy in tasks like image segmentation. Tests show their method helps transfer useful information between different medical imaging sources even when paired data is unavailable.
Multimodal LearningCross-Modal Knowledge DistillationUnpaired DataAlignment ModuleAttention PoolingOptimal TransportUncertainty WeightingPrototype Contrastive LearningSemantic SegmentationMedical Imaging
Authors
Rafsan Jany, Shadab Tanjeed Ahmad, Ahsan Bulbul, Tahsinul Islam, Md Azam Hossain, Abu Raihan Mostofa Kamal
Abstract
Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal distillation. Our approach introduces a backbone-agnostic Alignment Module that extracts semantically structured class tokens via an attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype memories to enforce global discriminability across unpaired batches. Evaluations on representative segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired datasets.