Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities
Computer Vision and Pattern RecognitionMachine Learning
Summary
The authors studied how to add a new type of medical image (Chest X-ray) to an AI model already trained on other types (Kidney CT and Brain MRI) without messing up its existing abilities. They found that simply adjusting the last layer of the model doesn’t work well for new image types, but tweaking convolutional layers helps a lot. They also showed that freezing a special low-rank convolutional basis learned from old images and only training small parts on new images can efficiently adapt the model with very few new parameters. Importantly, this method keeps old image performance intact, unlike full retraining which harms old skills. Lastly, they propose a way to detect when the model encounters a new image type so adaptation can be triggered at the right time.
Authors
Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim, Waseem Mustak Zisan, Hossain Md Shakhawat
Abstract
Deploying a medical imaging model that must later accommodate a modality it has never seen is a recurring practical problem: retraining the shared representation is expensive and destroys performance on the modalities already in service. We study this \emph{onboarding} problem under a strict leave-one-domain-out protocol, in which a convolutional backbone is pre-trained on source modalities (Kidney CT and Brain MRI), frozen permanently, and then required to accommodate an unseen modality (Chest X-ray). Under this protocol we establish three findings. First, decision-layer parameter-efficient fine-tuning is insufficient when the backbone has never observed the target modality: a linear probe and fully-connected LoRA both fall well short, whereas convolutional LoRA recovers most of the achievable accuracy, showing that adaptation must reach the convolutional features. Second, and centrally, the low-rank convolutional \emph{basis} learned on the source modalities \emph{transfers}: freezing that basis and training only its up-projections onboards the unseen modality using just $0.78\%$ of full fine-tuning's parameters, at an accuracy $6.11$ percentage points above a random basis of identical size, while an equivalent decision-layer basis exhibits no reliable transfer. Third, adapter-based onboarding leaves source-modality accuracy exactly unchanged ($Δ= 0.00$ pp), whereas full fine-tuning reaches the highest target accuracy only by catastrophically degrading the source modalities. A Mahalanobis score on frozen backbone features detects the unseen modality with high sensitivity at a strict source-retention threshold, providing a practical trigger for when onboarding is required. All results are reported over three seeds with paired bootstrap confidence intervals.