Extending TotalSegmentator: Predicting Patient and Acquisition Characteristics from CT and MR Images
Artificial Intelligence
Summary
The authors created computer models that can quickly figure out details about patients and how CT or MR scans were taken just by looking at the images themselves. They trained these models on many scans and tested them on new ones, where the models did a good job guessing things like the patient's weight, height, age, sex, and scanning settings. Their models worked better than other baseline methods and were fast enough to run in about 12 to 20 seconds. This helps when patient info is missing or unreliable in medical image databases.
3D ResNetConvolutional Neural Network (CNN)CT scanMRIMedical image metadataImage acquisition parametersMachine learningWeight predictionContrast agentImage noise
Authors
Jakob Wasserthal, Joshy Cyriac, Michael Bach, Kimia Mozahheb Yousefi, Minh-Son To, Máté Sik, Cédric Hémon, Thomas Weikert, Martin Segeroth
Abstract
Background: Patient details and acquisition metadata are important for clinical decisions, image quality control, and automated research pipelines, but may be missing or unreliable in imaging archives. Purpose: To develop and evaluate a fast open-source model that predicts patient and acquisition characteristics directly from CT and MR images. Materials and Methods: Separate 3D ResNet-10 ensembles for CT and MR were trained on 57,291 and 43,200 clinical examinations acquired from 2011 to 2025. Both predicted weight, height, age, sex, contrast presence, vertebral coverage, and image noise. The CT model additionally predicted scanner manufacturer, tube voltage, tube current, convolution kernel, and post-injection time; the MR model predicted sequence class. Performance was evaluated on internal CT (n=501) and MR (n=636) test sets and an external CT dataset (n=54). Results: Internal CT MAEs were 3.90 kg, 3.68 cm, and 4.42 years for weight, height, and age, with sex F1=0.990; corresponding MR results were 4.34 kg, 4.62 cm, 7.13 years, and F1=0.970. The CNN outperformed a segmentation-derived XGBoost baseline for all four core targets in both modalities (adjusted P<=.042). F1 scores were 0.963 for CT contrast, 0.953 for MR sequence, and 0.823 for MR contrast. External CT MAEs were 4.45 kg, 4.05 cm, and 5.17 years, with sex F1=0.971. CPU inference required 20 seconds for CT and 12 seconds for MR. Conclusion: One 3D multitask model per modality can rapidly recover patient and acquisition characteristics from heterogeneous CT and MR examinations. Models are available in TotalSegmentator: https://github.com/wasserth/TotalSegmentator