Multimodal AI improves Alzheimer’s stage and biomarker prediction with missing data
MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction
Computer Vision and Pattern RecognitionMachine Learning
Summary
Predicting how Alzheimer’s disease progresses is important but hard because patient medical data often come from many types of tests, and some data are missing. The authors present a new AI method called MMAP that learns from both images and other medical data together, even when some parts are missing. This method helps predict whether a patient’s condition will worsen and if certain disease indicators are present. MMAP works better than other approaches that use only one type of data or do not handle missing information well.
What this means in practice
- •For clinical ai developers: Create prediction models that remain accurate despite missing patient imaging or tabular data for Alzheimer’s disease progression.
- •For medical data engineers: Integrate multimodal encoders into processing pipelines that handle incomplete clinical datasets for better disease trajectory modeling.
Authors
Fiona Kekwick, Matthew Baugh, Bernhard Kainz, Paul M. Matthews, Wenjia Bai
Abstract
Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is characterised by multimodal medical data. AI holds great potential for learning useful representations from multimodal medical data to predict disease progression and aid clinical decision making. However, development of predictive AI models is constrained by missing modalities and incomplete tabular data frequently occurring in medical datasets. In addition, disease labels alone may only provide limited supervisory signals for learning representations from high-dimensional multimodal data. Here, we present MMAP, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data. An image encoder is pretrained with efficient sigmoid contrastive learning combined with generative reconstruction. A tabular encoder is built upon a tabular foundation model. A missing token generator enables the two encoders to take incomplete data as input, enabling the model to be robust against missing modalities, either with missing images or missing tabular data. We evaluate the clinical usefulness of the learnt multimodal representations on two challenging longitudinal clinical tasks for Alzheimer's disease: predicting disease stage conversion and predicting amyloid status. The proposed method outperforms strong multimodal and unimodal baselines.