Local variable importance measured with realistic data perturbations
LUCID: Local Understanding of Covariate Importance with Conditional Density Estimation
Machine Learning
Summary
Machine learning models often need explanations about which inputs most affect their predictions. Standard methods change one input at a time without considering how inputs relate to each other, leading to unrealistic examples and misleading explanations. The authors developed LUCID, which changes inputs in a way that respects how features normally co-occur, making the explanations more trustworthy. They showed that LUCID works well in both fake and real-world data tests and provided mathematical guarantees for its reliability.
What this means in practice
- •For data science teams: Improve trustworthiness of feature importance explanations by considering realistic feature relationships in model debugging and validation.
- •For healthcare data analysts: Assess variable impact locally on sensitive health prediction models while maintaining plausible patient data distributions.
Authors
Yixiao Wang, Srikar Katta, Jon Donnelly, Emanuele Borgonovo, Cynthia Rudin
Abstract
Computing local-level variable importance (VI) is critical for understanding machine learning models. Local permutation-based VI metrics offer a simple yet powerful approach, measuring how much a model's output changes when perturbing its inputs. However, existing perturbation strategies often ignore feature dependence or account for it only through restricted approximations. Perturbing a feature without fully respecting this dependence can generate unrealistic counterfactuals, thereby evaluating the model in regions unsupported by the observed data distribution. We address this gap by introducing a new model-agnostic local VI metric and estimator, Local Understanding of Covariate Importance with conditional Density estimation (LUCID). LUCID perturbs features of interest using plausible values drawn from the full conditional distribution given the remaining observed features. We establish theoretical connections between LUCID and existing VI measures, and provide an estimator with convergence guarantees and uncertainty quantification. Finally, we demonstrate LUCID's stable performance and practical utility across simulated and real-world examples.