Graph neural network improves multimodal retinal disease detection accuracy

Interpretable Context-Aware Multi-Label Retinal Classification with Dual-Hub Heterogeneous Graph Neural Network

Computer Vision and Pattern RecognitionMachine Learning

Summary

Detecting multiple eye diseases from retinal images is tricky because different conditions can appear together and patient information matters. The authors developed a special kind of graph-based AI that looks at both the images and clinical data together, clearly showing how each part contributes to the decision. Their method matched or beat existing image-only techniques on a big eye health dataset, especially improving detection of macular edema and diabetic retinopathy. This approach also helps doctors understand why the AI made each diagnosis.

What this means in practice

  • For medical imaging teams: Combine retinal photographs and patient clinical data to improve multi-disease eye condition screening with interpretable AI.
  • For clinical decision support developers: Build diagnostic tools that explain how specific retinal regions and clinical factors influence disease predictions for eye health professionals.$Commercial implications: Enables creation of transparent, multi-condition retinal screening products targeting healthcare providers seeking interpretable AI diagnostics.

Authors

Proshanto Chanda, Munib Mesinovic, Tingting Zhu, Abhirup Banerjee

Abstract

Multi-label retinal screening requires capturing co-occurring pathologies and clinical context, yet standard multimodal fusion remains opaque. We propose a dual-hub heterogeneous graph neural network (GNN) that couples spatial fundus features and structured clinical covariates through cross-modal message passing while providing inherent interpretability. Rather than relying on post-hoc saliency approximations, our architecture exposes multi-level decision pathways during forward aggregation: spatially biased self-attention captures lesion-level patch interactions, while modality-specific graph attention coefficients explicitly quantify the contribution of visual regions and clinical risk factors to each disease node. Evaluated on the BRSET dataset (16,266 photographs from 8,524 patients), the proposed model achieves 76.1% Macro-AUPRC across seven conditions, matching or exceeding image-only (75.9%) and ResNet-50+MLP fusion (75.9%) baselines, with largest contextual gains in macular edema (+3.5%) and diabetic retinopathy (+2.1%). Codebase is available at: https://github.com/proshanto-c/ca-mlc-refactor.