Large language models improve chest CT protocol selection accuracy
Automated Chest CT Protocol Selection via Large Language Model Derived Text Embeddings from Imaging Request Text
Machine Learning
Summary
Choosing the right CT scan settings is important for clear images and patient safety, but doing this by hand can be slow and inconsistent. The authors used a large language model to understand doctors' written requests more deeply and suggest the correct CT scanning protocols. Their system matched radiologists' accuracy on sample cases and worked well across many different scan types, sometimes even outperforming experts. This approach could help speed up and standardize how CT scans are prepared.
What this means in practice
- •For radiology departments: Automate chest CT protocol assignment from doctors’ free-text requests to reduce variability and improve workflow efficiency.
- •For medical imaging software developers: Integrate LLM-driven text understanding to enhance protocol recommendation features in radiology information systems.
Authors
Zahra Hosseini, Mahan Pouromidi, Farzad Khalvati, Patrik Rogalla
Abstract
Purpose: Accurate CT protocol selection is critical for diagnostic quality and patient safety, yet the current process is manual, time-consuming, and prone to inconsistencies. Prior Machine Learning methods using keywords or bag-of-words lack contextual understanding and perform poorly on rare protocols. We propose a decision support system using large language model (LLM) features to recommend protocols from free-text clinical indications, capturing clinical nuance and phrasing variation for more consistent, efficient selection. Methods: In this REB-approved retrospective study, 285,123 chest CT imaging requests from a large academic medical center (2017-2024) were split into training (228,099, 80%) and held-out test (57,024, 20%) sets. Each request included procedure names, clinical indication, HIS comments, and the selected protocol. Clinical text was embedded using a fine-tuned LLM, Meta's LLaMA-3.1-70B; these features input a logistic regression classifier predicting 18 protocol labels (e.g., PE, LDCT). Results: The pipeline achieved a weighted precision of 0.84, weighted F1-score of 0.81, and overall accuracy of 79% across 18 CT protocols. On 300 independent cases with expert consensus, the LLM reached an overall accuracy of 80% versus 83% for radiologists, with no significant difference (p = 0.263). Performance was comparable across most classes, with the LLM exceeding radiologists for some challenging categories, and entropy analyses indicated more balanced protocol use, suggesting reduced variability. Conclusion: An LLM-based recommendation system can leverage general knowledge from a large natural-text corpus to accurately assign chest CT protocols from free-text imaging requests, and may serve as a viable foundation for protocol recommendation tools where inputs require language understanding.