ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
2026-08-26 • Machine Learning
Machine LearningArtificial IntelligenceComputer Vision and Pattern Recognition
AI summaryⓘ
The authors address a problem where deep neural networks learn shortcuts by relying on accidental correlations in training data. Traditional methods check each concept, like patient sex, one by one, risking confusion when concepts overlap. They propose ICON decomposition, which measures how much each concept uniquely contributes to a model's decisions after factoring out other concepts. Tested on synthetic and real medical data, ICON better identifies true concept importance and provides clearer model explanations. This helps understand what a model really bases its predictions on.
deep neural networksshortcut learningconcept-based explainabilityICON decompositionvariance decompositionsynthetic dataskin lesion imagingbrain imagingmodel interpretabilityout-of-distribution testing
Authors
Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter
Abstract
Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning. Concept-based explainability methods screen for shortcuts by testing whether concepts such as a patient's sex or scanner settings can be decoded from a network layer. Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them. We introduce ICON decomposition, which instead quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing.