Class conditioned explanation guidance improves medical image analysis accuracy

Learning from the Right Regions: Class-Conditioned Explanation Guidance for Medical Imaging

Computer Vision and Pattern RecognitionMachine Learning

Summary

Sometimes AI models that analyze medical images make predictions based on the wrong parts of the image, which can be confusing or misleading. The authors developed a new approach that lets the AI focus on the right areas linked to each specific condition detected in an image, even when multiple conditions appear together. They tested their method with different AI systems and found the explanations became much better aligned with the correct regions, while the model’s prediction accuracy stayed about the same. This method helps make AI decisions in medical images more understandable and trustworthy.

What this means in practice

  • For medical imaging software developers: Build diagnostic tools that highlight the correct regions associated with multiple abnormalities to improve interpretation reliability.
  • For radiology it teams: Enhance AI-based imaging systems by integrating class-specific explanation guidance that better matches expert annotations.

Authors

Zubair Faruqui, Rahul Dubey

Abstract

Deep neural networks can achieve strong predictive performance in medical imaging while relying on regions unrelated to the predicted finding. Explanation-guided learning can address this problem, but multi-label classification introduces a key challenge: multiple abnormalities may coexist in an image, requiring each prediction to align with its corresponding spatial evidence. We propose a class-conditioned explanation-guided learning framework that constructs a separate explanation for each positive class and penalizes evidence outside its class-specific annotated regions. We instantiate the framework using Gradient-weighted Class Activation Mapping (Grad-CAM) and evaluate it on 14 abnormalities using EfficientNet-B1, ViT-B/16, ViT-B/32, and Swin-B. Explanation guidance substantially improves annotation alignment across all architectures: All-Saliency Precision increases from 0.140 to 0.625, 0.052 to 0.551, 0.060 to 0.403, and 0.095 to 0.570, respectively. These improvements are accompanied by smaller, architecture-dependent changes in classification performance. Our results show that class-conditioned explanation guidance can substantially improve where multi-label classifiers concentrate their explanatory evidence while largely preserving predictive performance.