Radiology report models improved by asymmetric diagnostic loss function
Diagnostic Loss: Asymmetric Supervision for Radiology Report Generation
Computer Vision and Pattern RecognitionMachine Learning
Summary
Generating automatic radiology reports is tricky because the model must clearly state correct medical findings and avoid mistakes. The authors point out that existing methods focus too much on matching words rather than on getting diagnoses right. They propose a new way to train these models using a special loss function called Diagnostic Loss, which encourages correct diagnoses to be emphasized and wrong ones to be suppressed. This method improves how well the reports reflect actual diagnoses without making the text less natural.
What this means in practice
- •For medical imaging software developers: Integrate Diagnostic Loss into radiology AI tools to produce reports that better match supported diagnoses from scans without extra runtime cost.$Commercial implications: Enables development of more accurate AI-based radiology reporting software that can be marketed to hospitals and diagnostic centers.
- •For healthcare data engineers: Improve training pipelines for radiology report generation models by applying the asymmetric Diagnostic Loss for enhanced diagnostic accuracy in existing datasets.
Authors
Ignacy Kolton, Wenxuan Li, Zongwei Zhou, Jarosław B. Ćwikła, Arkadiusz Sitek, Pedro R. A. S. Bassi, Szymon Płotka
Abstract
Automated radiology report generation requires models to do more than produce fluent clinical narratives: they must explicitly state supported diagnoses while avoiding unsupported ones. However, existing medical vision-language models (MedVLMs) are typically trained with token-level maximum likelihood objectives, which prioritize text agreement over diagnostically critical terms. As a result, models can miss key diagnoses and introduce wrong ones. We identify this as an objective misalignment between conventional language modeling and the asymmetric diagnostic requirements of radiology reporting. To encode these requirements, we introduce Diagnostic Loss (DLoss), a differentiable, architecture-agnostic objective that operates directly on decoder token probabilities. DLoss separately aggregates probability mass over vocabularies associated with diagnoses supported by the reference report and with unsupported alternatives. Using asymmetric pooling across token positions in the report, it encourages supported diagnoses to receive sufficient probability in at least one position while suppressing spurious probability spikes for unsupported diagnoses across all positions during training. The loss can be applied to existing 3D MedVLMs without modifying their vision backbones, language decoders, or inference procedures, and introduces no additional inference-time computation. We train and evaluate DLoss on 50,000 abdominal CT scans across several 3D MedVLM architectures on both in-distribution and out-of-distribution report-generation benchmarks. Across these settings, DLoss improves diagnostic faithfulness while maintaining competitive performance on conventional language-generation metrics, showing the value of asymmetric sequence-level supervision for radiology report generation. The code for DLoss is available at https://github.com/gmum/DLoss.