Model complexity and accuracy affect explainability in CNN image classification

How a Model's Architecture and Performance Influences Explainability: A Study of Grad-CAM and CRAFT in CNN-Based Image Classification

Computer Vision and Pattern RecognitionArtificial Intelligence

Summary

Deep learning models like CNNs are powerful but hard to understand, which raises concerns where trust is important. This study looks at how a model’s design and how well it performs affect the quality of explanations that help people see why the model made certain decisions. The researchers trained three types of CNNs from simple to complex and checked how explanations changed as the models got better. They found that more complex CNNs gave more trustworthy and stable explanations, while simpler models had easier-to-understand but less accurate explanations.

What this means in practice

  • For machine learning engineers: Design reliable explanation techniques by selecting CNN architectures that balance complexity and explanation quality during model training.
  • For healthcare ai teams: Improve trustworthiness of AI explanations in medical image analysis by choosing advanced CNN architectures with robust explanation characteristics.

Authors

Kilian Göller, Haadia Amjad, Steffen Seitz, Carsten Knoll, Ronald Tetzlaff

Abstract

Deep learning models, particularly convolutional neural networks (CNNs), are widely adopted in computer vision, yet their opaque decision-making raises concerns in high-risk applications like healthcare and autonomous systems, where trust and accountability are critical. Therefore, robust explainable AI (XAI) methods are desirable to provide transparent insights into model decisions. To ensure these methods are reliable and effective across diverse scenarios, it is important to understand how model performance and architectural complexity influence explanation quality. Our study takes an initial step toward systematically investigating the impact of these two factors on key XAI evaluation metrics. We constructed a pipeline using 10 subsets of ImageNet-1K, each containing 80 unique classes, on each of which we trained three CNN architectures of increasing structural complexity: VGG16, ResNet50, and ConvNeXt-Tiny. For each individual model, we analyzed 10 training checkpoints representing increasing levels of model accuracy using two XAI methods: Grad-CAM (feature-attribution-based) and CRAFT (concept-based). We assessed these methods across four evaluation categories: Faithfulness, Correctness, Sensitivity, and Simplicity. Our findings reveal that as training progresses, the complex architectures (ResNet50 and ConvNeXt-Tiny) exhibit improved Faithfulness and lower Sensitivity, indicating more robust explanations. In contrast, VGG16's simpler architecture yields simpler but less faithful and correct explanations.