Model explainability improved by finding when input regions cause outputs
Regional Explanations via Causal Sufficiency and Necessity
Machine Learning
Summary
Machine learning models are often hard to understand because they work with many features and decisions. This paper introduces a new way to explain models by finding specific input regions that are both necessary and sufficient for certain outputs to happen. The authors create a method to learn these regions and check their quality using ideas from probability. Their approach combines mathematical shapes and masks to keep explanations understandable and useful. Tests show that their method produces reliable and clear explanations for model predictions.
What this means in practice
- •For machine learning engineers: Identify specific input conditions that guarantee and are required for certain model predictions, improving model debugging and trust.
- •For data science teams: Provide clear regional explanations of model outputs to communicate insights and decisions to stakeholders more effectively.
Authors
Xuexin Chen, Peng Liang, Zijian Li, Zhiyong Lin, Ruichu Cai
Abstract
Model explainability is essential for understanding and trusting machine learning models. Existing explainable AI methods often explain predictions through feature importance, counterfactual explanations, or rules. However, a region-level characterization of when and only when a prediction behavior arises remains less explored. This paper proposes Causal Sufficient and Necessary Regional Explanations (SNRE), a framework that learns an input region $A$ and output region $B$ such that membership in $A$ is both sufficient and necessary for the model output to fall in $B$. Motivated by the classical Probability of Necessity and Sufficiency (PNS), we formulate a region-level PNS measure through stochastic interventions and derive a differentiable finite-sample estimator for optimization. SNRE parameterizes the input-output region pair with explicit and interpretable algebraic region families, together with a learnable feature mask, balancing expressiveness and interpretability. Experiments demonstrate that SNRE learns region pairs with strong sufficiency-necessity performance, robust explanation behavior, and practical utility for model analysis.