Large language models help choose the right chest CT scan protocol
Automated Chest CT Protocol Selection via Large Language Model Derived Text Embeddings from Imaging Request Text
Machine Learning
Summary
Choosing the correct CT scan settings is important for good medical images and patient safety, but doing this by hand can be slow and inconsistent. The researchers used a computer program trained on a huge amount of medical text to understand doctors' notes and pick the best CT protocol automatically. Their system was almost as accurate as expert radiologists at selecting the right protocol and worked well across many different types. This approach could make the process faster and more reliable by understanding the meaning behind doctors' free-form text requests.
CT protocolLarge language modelText embeddingChest CTMedical imagingMachine learningNatural language processingRadiologyLogistic regressionMeta LLaMA
Authors
Zahra Hosseini, Mahan Pouromidi, Farzad Khalvati, Patrik Rogalla
Abstract
Purpose: Accurate CT protocol selection is critical for diagnostic quality and patient safety, yet the current process is manual, time-consuming, and prone to inconsistencies. Prior Machine Learning methods using keywords or bag-of-words lack contextual understanding and perform poorly on rare protocols. We propose a decision support system using large language model (LLM) features to recommend protocols from free-text clinical indications, capturing clinical nuance and phrasing variation for more consistent, efficient selection. Methods: In this REB-approved retrospective study, 285,123 chest CT imaging requests from a large academic medical center (2017-2024) were split into training (228,099, 80%) and held-out test (57,024, 20%) sets. Each request included procedure names, clinical indication, HIS comments, and the selected protocol. Clinical text was embedded using a fine-tuned LLM, Meta's LLaMA-3.1-70B; these features input a logistic regression classifier predicting 18 protocol labels (e.g., PE, LDCT). Results: The pipeline achieved a weighted precision of 0.84, weighted F1-score of 0.81, and overall accuracy of 79% across 18 CT protocols. On 300 independent cases with expert consensus, the LLM reached an overall accuracy of 80% versus 83% for radiologists, with no significant difference (p = 0.263). Performance was comparable across most classes, with the LLM exceeding radiologists for some challenging categories, and entropy analyses indicated more balanced protocol use, suggesting reduced variability. Conclusion: An LLM-based recommendation system can leverage general knowledge from a large natural-text corpus to accurately assign chest CT protocols from free-text imaging requests, and may serve as a viable foundation for protocol recommendation tools where inputs require language understanding.