Efficient multi-level knowledge transfer improves radiology report generation
Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation
Computer Vision and Pattern Recognition
Summary
Creating detailed medical reports from X-ray images is tough because it requires precise medical knowledge. The authors designed a new method called EMGKT that helps computer models learn and use medical information at different levels of detail. This method teaches the model by mimicking expert knowledge during training but runs quickly without extra help when actually writing reports. It also uses specialized expert parts to better identify diseases in the images. The approach improves accuracy and works well with many existing methods.
What this means in practice
- •For radiology software developers: Integrate EMGKT to enable more accurate automatic generation of radiology reports from X-ray images with efficient fine-grained medical knowledge use.
- •For medical ai companies: Use EMGKT to create improved AI diagnostic tools that generate clinical descriptions while reducing computational costs during deployment.$Commercial implications: This enables product-level AI systems that produce professional radiology reports efficiently for hospitals and clinics.
Authors
Xubin Zhong, Zheyu Zhang, Wenjian Qin, Ning Wen
Abstract
Radiology report generation can automatically generate clinical descriptions from X-ray images, thereby significantly improving the efficiency of radiologists. This task is challenging because it requires medical knowledge to accurately identify diseases and describe them in a professional manner. However, existing methods often overlook the importance of enhancing medical knowledge in describing pivotal areas, a capability that requires models to effectively extract and aggregate knowledge at multiple levels of granularity. Accordingly, we herein propose a novel and compact Efficient Multi-Granularity Knowledge Transfer (\textbf{EMGKT}) method to address the above issues. First, we encode global knowledge embeddings using a medical vision-language model, which provides contextual medical knowledge. Moreover, we devise a novel Fine-Grained Knowledge Distillation (FGKD) training task which efficiently extract fine-grained knowledge. Specifically, the FGKD training task contains teacher embeddings and student embeddings. Teacher embeddings are encoded using extra priors; while student embeddings are learned from the teacher embeddings through knowledge distillation. During inference, the student embeddings are used to enhance fine-grained knowledge while the teacher embeddings are discarded, resulting in negligible computational costs and no need for extra priors. Finally, we further develop a mixture of disease diagnosis expert classifiers to enhance knowledge extraction. The classifiers are initialized using disease embeddings and are modeled as different experts to address various granularity features. Notably, \textbf{EMGKT} can be efficiently applied to most existing methods. Extensive experiments are conducted on two widely-used public datasets and various baselines, which demonstrates the effectiveness and transferability of \textbf{EMGKT}.