Position: Explanation Stability Is a Property of the Model Method Pair, Not the Model
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors argue that saying explanations from AI models are stable or reliable isn't scientifically valid unless tested with multiple explanation methods. They show that three different deep learning models all performed well on chest X-rays, but their explanation stability rankings switched depending on which explanation technique was used. For example, one method rated InceptionV3 as most stable, while another favored DenseNet201 and gave InceptionV3 a much lower score. This means explanation stability depends on both the model and the explanation method together, not just the model alone. The authors suggest that claims about explanation reliability should always be tested across different methods and clearly state which explanation method was used, especially in safety-related contexts.
Authors
Kabilan Elangovan, Daniel Ting
Abstract
This position paper argues that claims about explanation stability are scientifically invalid without cross method validation. Just as statistical significance requires the test statistic to be specified, stability should either be evaluated across multiple attribution paradigms or explicitly scoped to the computational objective of a single method. In controlled chest X ray experiments, DenseNet201, ResNet50V2, and InceptionV3 achieved AUC values above 99%, yet their stability rankings reversed across attribution methods. LayerCAM ranked InceptionV3 as the most stable model, with an IoU of 0.777, whereas GradCAM++ favored DenseNet201 and reduced InceptionV3 stability score by 17.3%. These findings demonstrate that explanation stability is an emergent property of the model method pair rather than an intrinsic characteristic of the model alone. We therefore argue that explanation based claims should be validated across multiple attribution methods and that regulatory submissions should explicitly specify the attribution operators used to avoid creating illusory safety assurances.