Hide&Seek: Learning to Explain in an End-to-End Differentiable Network
2026-08-17 • Machine Learning
Machine Learning
AI summaryⓘ
The authors propose Hide&Seek, a new method for explaining predictions by selecting important features for each individual data point. Unlike earlier methods, their model trains quickly and avoids common problems like information leakage by smoothly adjusting feature importance instead of simply turning features on or off. They combine feature selection and prediction into a single, end-to-end learning process, making the approach more accurate and efficient. Experiments show Hide&Seek performs better than previous techniques.
instance-wise feature selectionblack-box modelsinformation leakagedifferentiabilityend-to-end learningfeature removalparsimonyannealingpredictorselector
Authors
Tal Ellinson, Hadi Mohasel Afshar, Sally Cripps
Abstract
Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.