Whole-Body MRI Classification via Prompt-Based Clinical Conditioning
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new method called TACTIC that helps combine whole-body MRI scans with patient information from clinical data to better diagnose diseases. Unlike previous methods, TACTIC can handle missing or incomplete clinical data without needing to fill in gaps beforehand. They tested this method on five different disease detection tasks and found that it improved diagnosis accuracy when clinical data was available, while still working well when some data was missing. This approach uses a special way to include patient data as prompts that guide the MRI image analysis.
whole-body MRIclinical variablesmultimodal fusionprompt-based learningtransformer modelstabular datamissing data handlingimage classificationsystemic diseasesoncologic diagnosis
Authors
Laura Daza, Marta Hasny, Cristina González, Julia A. Schnabel
Abstract
Combining whole-body magnetic resonance imaging (WB-MRI) with clinical variables has the potential to improve systemic disease diagnosis by leveraging complementary sources of patient information. However, structured clinical variables are often incomplete or missing, limiting the applicability of conventional multimodal fusion methods that assume fixed inputs. In this work, we propose TACTIC (Tabular-Attribute Conditioned Transformer for Image Classification), a prompt-based multimodal framework that integrates WB-MRI and structured clinical data through conditional visual feature learning. By encoding clinical attributes as prompts, TACTIC supports an arbitrary number of tabular inputs and naturally handles missing data without requiring imputation or fixed input structures. We evaluate TACTIC on five WB-MRI classification tasks spanning systemic and oncologic applications, including diabetes, chronic obstructive pulmonary disease (COPD), breast cancer, prostate cancer, and metastasis diagnosis. Across all tasks, TACTIC consistently improves performance over image-only baselines when clinical information is available while maintaining strong predictive capability under incomplete tabular inputs. Our results demonstrate the effectiveness of prompt-based models as a flexible approach for improving WB-MRI analysis using clinical context. The model weights and code are available at https://github.com/lauradaza/TACTIC