Task specific adapters improve medical image analysis accuracy and speed
Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Medical images come in many types and require different analysis tasks like classification, counting, or generating reports. The authors designed a system that uses one main AI model with small extra modules, called adapters, customized for each task and imaging type. This approach improves accuracy and efficiency without needing to retrain the whole model every time. Their system was tested on a large, diverse set of medical images and achieved good performance on multiple tasks.
What this means in practice
- •For medical imaging engineers: Build more accurate and efficient medical image analysis systems by integrating task-specific low-rank adapters into a shared vision-language model.
- •For clinical ai developers: Deploy adaptable AI pipelines capable of parsing various medical imaging modalities and tasks without retraining the full model for each case.
Authors
Taha Sabir, Alberto Montero Molina, Thinh Quang Ly, Veronika Dmitrenko, Mohammad Yaqub, Tausifa Jan Saleem
Abstract
Adapting a single vision-language model to medical image parsing requires training choices that reflect the demands of each task. We study task-specific adaptation with Qwen3.5-9B for FLARE 2026 Task 3, covering six task families and seven imaging modalities. Our system couples a shared, frozen 4-bit backbone with LoRA adapters selected from the input question. We customize adapter ranks and prompts, rebalance rare classification targets, and supplement regression training with dermatology classification examples. On the fixed internal holdout, the task-specific adapters achieve a dataset-macro balanced accuracy of 0.7791 for classification, a dataset-macro micro-F1 of 0.6286 for multi-label classification, a counting mean absolute error (MAE) of 11.73, a dataset-macro detection F1 of 0.8914, a dataset-macro regression MAE of 9.78, and a report-generation CRIMSON score of 0.8204. On the organizer's 1,703-example hidden-validation set, the submitted system achieves 0.8496 classification balanced accuracy, 0.5338 sample-averaged multi-label F1, and 14.5950 regression MAE. The final 30.1 GB container averages 3.1 seconds per case, including model loading, and produces identical outputs across repeated verification runs. The benefits vary across tasks, supporting the use of task-specific adapter configurations and image budgets within a shared medical vision-language model. Code is available at https://github.com/TahaSabir0/FLARE26-Hexapus.