Style aware fine tuning improves automated radiology report writing
Corpus Characterization and Inverse Constitutional Fine-Tuning for Style-Aware Radiology Reports
Computation and Language
Summary
Automatically generated radiology reports often look different from those written by real doctors, which can make them less trustworthy. The authors studied thousands of real reports to find common styles in how radiologists write. They then helped a language model learn to write reports that match these styles by teaching it with a special method that focuses on tone, word choice, and report structure. This approach made the model produce reports that look much more like those written by real radiologists.
What this means in practice
- •For medical software developers: Improve automated radiology reporting software to generate reports matching authentic radiologist writing styles and terminology accurately.
- •For healthcare data teams: Enhance clinical documentation systems by integrating style-aware models that better calibrate uncertainty and narrative structure in reports.
Authors
Sarah Y. Li, Elijah Renner, Rayan Ansari, Alaa Youssef
Abstract
Automated radiology report generation has advanced rapidly in diagnostic accuracy, yet generated reports frequently diverge from the stylistic conventions of authentic radiologist writing in structure, diction, and uncertainty language, a gap which has direct implications for clinician trust and user experience. To address this, we characterize stylistic variation across 2,000 reports from the CheXpert Plus dataset using Bio-ClinicalBERT embeddings, UMAP dimensionality reduction, and HDBSCAN clustering, identifying five distinct reporting patterns differing in pathology focus, narrative structure, and lexical preference. Drawing on these findings, we adapt the inverse constitutional AI framework to derive a style-focused constitution from radiologist-written report pairs without requiring a formal preference dataset. This constitution, encoding conventions of tone, diction, uncertainty calibration, and report structure, is incorporated into the supervised fine-tuning of a MedGemma-4B base model on 25,245 CheXpert Plus training pairs. Constitutional fine-tuning produces a substantial increases in text alignment (BLEU-4: 0.006 to 0.308; ROUGE-L: 0.171 to 0.484) relative to the untuned baseline. These gains show a qualitative shift in structural and lexical alignment rather than marginal improvement, as the baseline model produces near-zero scores due to format mismatch. Overall, we establish corpus-level style characterization and constitutional modeling as an effective and data-efficient strategy for producing radiology reports that conform to authentic radiologist writing conventions.