Explanation reliability declines under changing data but stays relevant
Does Explainability Survive Data Drift?
Machine LearningArtificial Intelligence
Summary
Models in machine learning often lose accuracy when the data they see changes, but explanations of their decisions are not usually checked as carefully. This paper shows that explanations can become less accurate even if the relationship between the input data and results stays stable. The authors tested this on a fraud detection dataset and found that older explanations still matter but explain less precisely compared to new ones. The study suggests explanations should be monitored regularly and always tied to the specific version of the model that created them.
What this means in practice
- •For financial fraud teams: Ensure fraud detection explanations remain valid over time despite data changes by monitoring explanation fidelity separately from performance.
- •For healthcare ai developers: Maintain trustworthy patient outcome explanations by retraining models and updating explanations as data distributions evolve.
Tested on one dataset.
Authors
Samuel Ozechi
Abstract
Model performance monitoring is a standard practice in machine learning deployments. Detection performance is tracked continuously, and model decay is expected as the relationship between the feature and target variables degrades, a phenomenon known as concept drift. Explanation fidelity, however, is rarely monitored with the same discipline, even in domains such as financial systems, healthcare, and other regulated environments where explanations are required for governance purposes. This paper investigates whether explanations can decay under data drift, even when the feature-target relationship remains stable, and whether explanations produced before drift occurs remain faithful to the decisions of the model that replaces them. Using the IEEE-CIS Transaction Fraud Detection dataset, we find statistically significant covariate shift but no statistically significant evidence of concept drift under the implemented conditional-drift tests, thereby providing an empirical setting in which input distributional change can be studied separately from detectable changes in the feature-target relationship. Local explanations are generated with ExIFFI and evaluated at three levels: path validity, structural behaviour, and fidelity under controlled intervention. Results show that while prior explanations retain substantial decision relevance to a retrained model, they are consistently less faithful than newly generated explanations, with no evidence of a systematically widening gap across the evaluated windows. The study shows that explanation fidelity requires its own monitoring, that structural stability of explanations does not guarantee functional fidelity, and that explanations should be treated as artifacts tied to the model that produced them.