Medadaptagent automates tuning medical imaging models for new hospitals
MedAdaptAgent: An Automatic Harness for Adapting Medical Imaging Models to New Sites
Computer Vision and Pattern Recognition
Summary
Models trained to analyze medical images often perform worse when used at different hospitals due to differences in how images are taken or processed. The authors introduce MedAdaptAgent, a system that automatically adjusts existing medical image analysis models to work better with new hospitals' data. MedAdaptAgent uses large language models to plan adaptation steps and checks improvements before delivering an updated model package. Tests show the system greatly improves performance on tasks like identifying structures in images and classifying conditions, even with limited new data.
What this means in practice
- •For hospital data teams: Automatically adapt pretrained medical image models to local imaging protocols, improving diagnostic tool performance with minimal new labeled data.
- •For medical device developers: Package and deliver updated medical imaging AI models tailored to new clinical sites, ensuring reproducible and verified improvements.$Commercial implications: Enables selling adaptable AI software packages customized for hospitals, reducing integration effort and improving reliability.
Authors
Hanxue Gu, Zhonghao Liu, Xingrui Gu, Kang Wang, Yang Yang
Abstract
Reusing pretrained medical-imaging models offers an alternative to training from scratch. A model trained for the same task may perform poorly at a new hospital because imaging protocols and preprocessing differ. Effective adaptation depends on the model architecture, domain shift, and available labels and compute. Adaptation can improve scores on a small labeled set yet reduce performance on unseen data. We present MedAdaptAgent, an automatic harness that turns a supplied model, target data, and a free-text request into a new-site adaptation workflow. Its LLM-driven Adaptation Planner uses model-site evidence to choose compatible experiments and revise subsequent adjustments. The harness requires evidence of improvement over the pretrained model, then delivers the selected model as a complete, runnable package. Evaluation spans 26 segmentation and classification transfers across architectures and compute settings, with segmentation annotation budgets ranging from zero-shot transfer to few-shot adaptation. With 20 local annotations, mean segmentation Dice rises from 0.483 to 0.774 across 10 transfers, a 60.1% relative gain, with no harmful replacements. Mean classification AUROC rises from 0.810 to 0.837 across 13 transfers, a 3.4% relative gain. Comparing adapted models with the pretrained model reduces harmful classification replacements by 30.8% versus selecting the highest-scoring model on the available labels, using identical candidates. Seven public-model deployments demonstrate adaptation and reproducible package delivery across contrast and modality shifts. A CT-to-MRI study extends the evaluation to four LLM planners, including two open-weight models. We release the code on GitHub.