Captor model improves cancer CT scan interpretation accuracy
CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Interpreting CT scans of the chest, abdomen, and pelvis for cancer can be challenging. The authors created CAPTOR, a computer model trained on many cancer-related CT scans and reports from around the world. CAPTOR learned to connect images and text to better detect disease, predict survival, and find similar scans. It performed better than other top models on several important cancer-related tasks. The authors also shared their code and model to help others build on their work.
What this means in practice
- •For radiology teams: Use CAPTOR to improve automated detection and prognosis of tumors in chest-abdomen-pelvis cancer CT scans.
- •For medical imaging software developers: Incorporate CAPTOR’s pretrained model into tools for faster and more accurate retrieval of oncologic CT images based on visual and textual similarity.
Authors
Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali, Amandine Crombé, Nathalie Lassau, Alexandre Bône
Abstract
Purpose : To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. Materials & Methods : CAPTOR (Chest-Abdomen-Pelvis foundaTion model for Oncologic Radiology) is a contrastive vision-language model, trained using a pretraining strategy for oncologic CAP CT (in single- or multiple-series acquisitions), on a multicenter retrospective dataset from North America, Europe and Asia, predominantly from tumor lesion examinations. CAPTOR was compared with state-of-the-art CT foundation models on three downstream tasks: disease detection, survival prediction using TumorImagingBench and image retrieval. Classification and retrieval were both evaluated on held-out internal and public test sets, stratified by anatomical coverage and indication. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results : CAPTOR was trained and tested on 25,512 and 10,794 CT-report pairs respectively. TumorImagingBench datasets comprised between 134 and 421 studies each. CAPTOR outperformed all baseline models, on the classification task, on the CAP subset (AUC 0.767, 95%CI: 0.734-0.793, vs. 0.695, 95%CI: 0.645-0.736 for the next-best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759-0.805, vs. 0.734, 95%CI: 0.703-0.767 p<0.001), ranked first on three of the four oncologic prognostic benchmarks, and achieved superior retrieval performance on CAP and oncologic subsets. Conclusion : A foundation model trained on oncology-enriched CAP CT data outperformed relevant state-of-the-art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and inference code : https://github.com/Guerbet-AI/CAPTOR and model weights : https://huggingface.co/Guerbet-AI/CAPTOR