Dual-input AI improves bone tumor diagnosis from X-ray images
Integrating Local Detail and Global Context: A Dual-Input Multi-Task Learning Framework for Bone Tumor Diagnosis
Computer Vision and Pattern Recognition
Summary
Looking at X-rays to diagnose bone tumors is tricky because tumors can look very different and blend with surrounding bones. The authors created a computer program that looks both closely at the tumor area and at the entire X-ray to better understand what's going on. Their program combines these two views to identify and classify tumors more accurately than older methods. It worked especially well at spotting dangerous types like malignant osteosarcoma, which could help doctors make earlier and more accurate decisions.
What this means in practice
- •For radiology departments: Assist radiologists in accurately segmenting and classifying bone tumors from X-rays using a system combining lesion detail with full-image context.
- •For medical imaging companies: Incorporate dual-input attention architectures into imaging software products to improve early detection and subtype diagnosis of bone tumors.$Commercial implications: Enables development of advanced diagnostic tools for bone tumor assessment that can be sold to hospitals and clinics.
Authors
S. M. Nasif Uddin, Rusab Sarmun, Muhammad E. H. Chowdhury, Adam Mushtak, Israa Al-Hashimi, Sohaib Bassam Zoghoul
Abstract
Primary bone tumors are rare but clinically aggressive neoplasms whose diagnosis from radiographs is challenged by heterogeneous morphology, subtle lesion margins, and overlapping bone structures. To address the limitations of existing single-view models, we present a dual-input, multi-task learning framework that, to our knowledge, is the first to apply bidirectional cross-modal attention between a lesion crop and the full radiograph for joint segmentation and subtype classification. Using the multi-institutional Bone Tumor X-ray Radiograph Dataset (BTXRD, n=3,746), we employ a YOLO-based detector to generate regions of interest, which are paired with full images as inputs to a dual-stream DenseNet121 architecture. Features are integrated via a novel cross-modal attention fusion strategy, refined by Hierarchical Multi-scale Feature Fusion, effectively balancing fine-grained lesion detail with global anatomical context. Evaluated on a held-out patient-level test split, the model demonstrates superior performance over single-input baselines, achieving an overall Dice Similarity Coefficient of 0.896 and a macro-averaged classification F1-score of 0.928. Notably, the system exhibits exceptional sensitivity for malignant osteosarcoma (AUC 0.999), validating the potential of dual-stream context modeling to support radiologists in accurate, early decision-making.