Contrastive adapter training improves skin cancer diagnosis across clinics
Contrastive Adapter Training and Consensus Knowledge Distillation for Multi-Source-Free Domain Adaptation in Skin Cancer Diagnosis
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Skin cancer diagnosis is hard because different cancer types look very similar. The authors found a new way to train AI models that helps them work better on skin images from different hospitals, without needing patient data from those hospitals. They use a special training method to make the AI focus on features that are common across sources and combine knowledge from multiple models more safely. This method improves overall accuracy while keeping models smaller and easier to train.
What this means in practice
- •For clinical ai developers: Improve skin lesion classification models to work well on images from new hospitals without accessing their original patient data.
- •For healthcare it teams: Deploy AI systems for skin cancer detection that maintain accuracy across multiple clinics while respecting privacy regulations.
Authors
Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah
Abstract
Skin cancer diagnosis, particularly differentiating melanoma from benign nevi, is challenging due to the visual similarity between lesions. Although convolutional neural networks (CNNs) and vision transformers (ViTs) have shown promising performance, their effectiveness often deteriorates on data from heterogeneous clinical sources. Conventional domain adaptation methods address domain shift but require access to source data during adaptation, which is often infeasible due to privacy regulations. Multi-source-free unsupervised domain adaptation (MSFDA) addresses this limitation by leveraging multiple labeled source domains to generalize to an unlabeled target domain without requiring access to source data. However, existing MSFDA methods rely on full backbone fine-tuning, leading to catastrophic forgetting and overfitting on small clinical datasets, and address domain shift at the aggregation stage without establishing a shared domain-invariant feature space. Furthermore, their reliance on hard pseudo-labels or confidence-weighted aggregation introduces noisy supervision signals under domain shift. To address these limitations, we propose CAT-CKD, consisting of two components: (1) contrastive adapter training (CAT), which trains lightweight ConvPass adapters within a frozen ViT backbone using supervised contrastive learning (SCL) to establish a shared domain-invariant feature space before source-specific model training, and (2) consensus knowledge distillation (CKD), which aggregates logits from multiple source models into a consensus supervisory signal and adapts a student model on unlabeled target data using KL divergence. Experiments on five publicly available skin lesion datasets show that CAT-CKD achieves an average AUROC of 86.1%, outperforming existing MSFDA methods while requiring only 4.3M trainable parameters. The code is available at https://github.com/A-Abedi/CAT_CKD.