Large language models as synthetic clinical experts to inform longitudinal rare-disease modeling
2026-08-17 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors used large language models (LLMs) as stand-ins for clinical experts to help analyze complex medical data for a rare disease affecting children’s motor skills. They asked the LLMs to give clinical opinions on patient data, then trained a model that summarizes this data while keeping the clinical meaning intact. This method helped the model better match real medical categories and improved predictions about patient progress compared to methods that didn’t use expert knowledge. Their approach shows a way to include expert insight into machine learning even when real experts are not available.
longitudinal datarare diseaselarge language models (LLMs)variational autoencoderlatent representationclinical knowledge elicitationsynthetic expertmotor function assessmentspinal muscular atrophymixed-effects model
Authors
Clemens Schächter, Astrid Pechmann, Janbernd Kirschner, Jan Hasenauer, Harald Binder
Abstract
Due to the limited amount of information, modeling longitudinal rare-disease data can benefit from integrating clinical knowledge. Yet, elicitation of expert knowledge and formalization for model fitting is challenging, in particular due to limited time of clinical experts. To nevertheless make domain knowledge accessible during model fitting, we use large language models (LLMs) as synthetic clinical experts to supervise a variational-autoencoder-based approach that learns low-dimensional latent summaries of visit-level observations. Specifically, LLMs are queried offline on textual descriptions of patient observations to obtain judgments, e.g., the suspected clinical category. To improve the variational autoencoder fit, we train a differentiable surrogate model on these judgments and augment the loss function to encourage reconstructions that preserve the clinical-label distribution of their corresponding input profile. In an application to longitudinal motor-function assessments from children with spinal muscular atrophy, we map visit-level clinical profiles to low-dimensional representations that are linked by a multivariate mixed-effects model. The synthetic expert loss discourages reconstructions that remain numerically close in data space but alter the clinical interpretation of the reconstructed motor function profile, such as by crossing a disease-type boundary. We thus reduced disagreement between original and reconstructed SMA type labels from about 11 to 7 percent. Furthermore, informing the latent representation by the synthetic expert improved prediction of motor function milestones compared with unsupervised latent representations and a data-level baseline. These results suggest that incorporating LLMs into model fitting can make clinical knowledge available to representation learning and improve clinical faithfulness for longitudinal rare-disease data.