Feature importance clarified by null importance for interpretable machine learning
Null importance: Disentangling relevance for interpretable machine learning
Machine Learning
Summary
Feature importance tells us how much a particular feature matters in a machine learning model, but the idea of "importance" can mean different things. The authors clarify this by introducing the idea of null importance, which describes when a feature is truly irrelevant according to different scientific questions. They explore different ways to define this irrelevance and show how these definitions matter in real-world cases like fairness in algorithms and genomic data analysis. Their work helps connect the questions scientists ask with the assumptions they make about data and the methods used to analyze features.
What this means in practice
- •For machine learning fairness teams: Use different definitions of feature irrelevance to better align fairness criteria with specific fairness goals in algorithmic decision-making.
- •For genomic data analysts: Determine which genomic features a prediction model actually learns by applying distinct notions of feature relevance to genomic perturbation data.
Authors
Garvesh Raskutti, Kris Sankaran, Jiaxin Ye
Abstract
Feature importance is central to interpretable machine learning, but the term "importance" encompasses several fundamentally different notions of relevance. We develop a unified perspective based on null importance: a population-level characterization of when a feature is irrelevant under a specified notion of relevance. We consider standard notions of null importance arising from marginal and conditional statistical relevance, predictive risk, functional invariance, and causal effects, and show how these notions answer different scientific questions. We illustrate the framework in two applications in which the distinction is particularly consequential: algorithmic fairness, where common fairness criteria correspond to different notions of null importance, and genomic perturbation modeling, where different notions of relevance lead to different conclusions about what a prediction model has learned. The framework connects three aspects of feature analysis: the scientific question defining relevance, the data and model assumptions that shape how different null notions relate, and the methods used to assess importance. We establish sufficient conditions under which null notions coincide and give counterexamples showing how they diverge when those conditions fail. We then characterize which nulls different method families target and when their zero-importance statistics identify those targets. Finally, simulations spanning feature dependence, redundancy, nonlinearity, hidden features and other standard phenomena, along with case studies on image and multiomics data, provide empirical evidence for these theoretical distinctions and their practical consequences. Taken together, these results provide a common statistical language for relating scientific questions, data-generating assumptions, and algorithms, and clarify the conclusions that feature-importance analyses can support.