End to end method boosts alzheimers diagnosis using images and data
M$^2$PFN: End-to-End Disentangled Alignment for Generalizable Multimodal In-Context Learning in Alzheimer's Disease
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Diagnosing Alzheimer's disease accurately across different patient groups is challenging when combining brain images and health data. The authors present a new approach called M²PFN that adapts a model originally designed for interpreting tabular data to work well with both images and tabular patient information. This method trains the system jointly so it learns to represent both types of data in a way that improves prediction accuracy. It was tested on multiple datasets and showed better performance than other methods, even when applied to new patient groups without retraining.
What this means in practice
- •For clinical data teams: Improve Alzheimer's diagnosis models by integrating brain images and patient data with a multimodal approach that generalizes to new hospitals without retraining.
- •For medical imaging developers: Develop integrated software tools that combine MRI scans and clinical measurements to provide more accurate Alzheimer's assessments in diverse patient populations.
Authors
Lujia Zhong, Shuo Huang, Jianwei Zhang, Xinyu Nie, Yonggang Shi
Abstract
While various multimodal methods combining imaging and tabular data for Alzheimer's disease (AD) diagnosis were proposed, they are often limited in generalization across cohorts. In-context learning (ICL) has demonstrated excellent generalization performances and high flexibility in foundational tabular models such as TabPFN. To extend TabPFN's ICL to multimodal AD analysis, the main obstacle is that TabPFN is meta-trained on synthetic tabular priors that do not naturally match the statistical structure of image-derived features. We propose M$^2$PFN, an end-to-end framework that turns this tabular foundation model into a multimodal AD predictor. M$^2$PFN (i) performs differentiable inference through TabPFN's transformer, back-propagating task gradients into 3D-MRI and tabular encoders; (ii) aligns the two modalities into a shared subspace, via disentanglement and a contrastive objective, matched to the ICL engine's prior; and (iii) folds in a frozen tabular-only prediction through a learnable gated shortcut. Because the ICL engine stays frozen, its in-context mechanism is preserved for test-time generalization, while end-to-end training shapes the encoders into features it can exploit. On ADNI ($n=2240$, three-class CN/MCI/AD), M$^2$PFN attains $65.55\%$ macro-F1 and $82.21\%$ macro-AUC, surpassing a comprehensive set of unimodal and multimodal baselines. By swapping only the head for a TabPFN regressor, the same architecture regresses baseline MMSE on a $1250$-subject sub-cohort to test MAE $1.743$, outperforming every multimodal baseline. On two external cohorts (OASIS-3 and SCAN) with no retraining, M$^2$PFN achieves the best AUC and the lowest MMSE MAE across all baselines, and transfers even when the cognitive instrument changes.