Nv-reason-ct combines 3d ct images and language for detailed analysis
NV-Reason-CT: 3D Visual Language Model for CT Analysis
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Analyzing 3D CT scans (detailed X-ray images of the chest and abdomen) is complex and time-consuming for doctors. The authors designed a computer model, NV-Reason-CT, that uses both the 3D images and expert-written text to better understand and explain scan findings. This model keeps the full 3D spatial information and links it with medical language to classify abnormalities, generate reports, and provide reasoning. It was tested on multiple datasets and helped radiologists reduce their review time while increasing confidence.
What this means in practice
- •For radiology departments: Use the model to assist radiologists in faster and more confident CT scan interpretation with integrated 3D image-language analysis and reasoning.
- •For medical software developers: Integrate the stable and open-sourced NV-Reason-CT model into products for automated CT report generation and differential diagnosis support.$Commercial implications: Enables medical imaging software companies to offer advanced, AI-powered CT analysis features to hospitals and clinics.
Authors
Andriy Myronenko, Dong Yang, Yucheng Tang, Baris Turkbey, Benjamin Simon, Stephanie Harmon, Rikhil Makwana, Mariam Aboian, Sena Azamat, Ibrahim Ethem Hamamci, Sezgin Er, Bjoern Menze, Marc Edgar, Yufan He, Pengfei Guo, Daguang Xu
Abstract
We present NV-Reason-CT, a generative vision--language model for chest and abdominal CT combining native 3D visual encoding with radiologist-guided reasoning. The model couples a native 3D vision transformer with a language model, passing all visual tokens and their explicit 3D coordinates into language decoding without further spatial token merging. This retains volumetric spatial information within the vision encoder and through the language model's positional encoding during joint processing with text. We train on a curated corpus of approximately 550,000 multimodal instruction examples from 70,111 unique CT image inputs, combining standardized reports, abnormality-focused and anatomy-specific questions, multi-turn interactions, and radiologist-authored reasoning from recorded and transcribed expert CT interpretations. Expert annotations provide direct supervision and guide additional report-grounded synthetic reasoning. End-to-end supervised fine-tuning (SFT) is followed by Group Relative Policy Optimization (GRPO), with verifiable rewards over chest and abdominal abnormality sets. The model supports abnormality classification, report generation, and interactive reasoning with reviewable observations, differential diagnoses, and uncertainty. Evaluation spans public CT benchmarks and a held-out NIH cohort. On CT-RATE, NV-Reason-CT achieves a macro-F1 of 0.614 and macro-AUROC of 0.871 without a task-specific classification head; generated reports achieve a report-derived macro-F1 of 0.592. In a preliminary study with expert radiologists, AI-assisted review received favorable confidence ratings and was associated with a 50% reduction in average reported interpretation and reporting time. We release the model and training code to support reproducible research on explainable AI for volumetric medical imaging.