AI summaryⓘ
The authors explain that current methods for making algorithm decisions understandable often confuse people because they require expert knowledge and may not actually reveal how the algorithm works. To fix this, they propose "Explanation Cards" that add extra details about how reliable and valid the explanations are, along with clear guidance on how to interpret them. This approach helps users better understand the explanations and shifts the responsibility to the explanation providers to make things clear. They show how this works using examples like counterfactual explanations and SHAP, and suggest that Explanation Cards could support legal requirements for AI transparency in the EU. Overall, the authors offer a tool to improve the practical use of algorithmic explanations in real life.
Algorithmic explanationsRobustnessValidityCounterfactual explanationsSHAPInterpretabilityExplainabilityEU AI ActExplanation algorithms
Authors
Eric Günther, Balázs Szabados, Kristof Meding, Gunnar König, Sebastian Bordt, Ulrike von Luxburg
Abstract
Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explanations is often not what one might intuitively expect, so expert knowledge is required to interpret them correctly. Second, recent work has shown that popular explanation algorithms are uninformative about the behavior of complex decision functions. Together, these issues create a gap between what explanations appear to convey and what they actually provide. In this work, we propose Explanation Cards for Explanation Algorithms, which augment standard explanations with complementary information about robustness and validity, as well as clear instructions for interpretation. The complementary information can render otherwise uninformative explanations practically useful, while also helping to detect cases where they are not. Importantly, the interpretation instructions in explanation cards shift responsibility from users to providers: Rather than expecting users to recognize what can and cannot be concluded from an explanation, providers must make this explicit upfront. Using counterfactual explanations and SHAP as examples, we demonstrate how providers can construct explanation cards and that these cards provide users with the guidance needed for sound interpretation. We further argue that explanation cards offer a practical means of operationalising the explainability provisions of the EU AI Act. Overall, explanation cards are a significant step toward making explanation algorithms fit for real-world use cases.