Papers for

machine learning interpretability teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

New method reduces distraction effects in image AI explanations

Seeing Through Distractions: Context-Stable Attribution via the Least Core

Abstract: Shapley-value-based explanations are widely used for feature attribution in machine learning. However, in computer vision, a feature may reduce the target-class score when added to the complete genuine image, even when the predicted label remains unchanged. We formalize such features as contextual distractors and investigate their attributions under semivalues and the least core. Under a weak average marginal monotonicity assumption, we derive quantitative semivalue--least-core gap bounds and sufficient conditions based on the probability of sampling smaller contexts. We further show that shifting semivalue weights toward larger contexts decreases the signed gap. Finally, controlled image-classification experiments and evaluations on Salient ImageNet provide strong empirical support for our theoretical findings.

Mon 26 OctComputer Vision and Pattern RecognitionComputer Science and Game Theory
The gist
Sometimes, parts of a picture can seem unhelpful or distracting when AI tries to explain why it made a decision, even if those parts don't change the final answer. The authors studied these distracting parts, called contextual distractors, and found ways to better measure their importance by looking at different sized groups of features. They tested their ideas with experiments and found their method helps make explanations clearer and more reliable. This could help people understand AI decisions in images much better.
Open → 2610.35827v1

Language models organize answers into shared concept directions

The Answer-Basin Representation Hypothesis: We Are Not Probing or Steering Concepts

Abstract: The Linear Representation Hypothesis associates high-level concepts with directions in language models, but it remains unclear how these concept-related linear structures are organized within the model. We propose the Answer-Basin Representation Hypothesis: the probability measure induced over answers by the model's continuation distribution organizes these linear structures, with its statistics represented along linear directions shared across questions. All continuations yielding the same answer form an answer basin, whose mass is their total probability. These basin masses define the pushforward probability measure over answers. We posit that concept-related linear structure emerges from differences in the answer measure rather than being determined by changes in concept labels. Experiments across models and tasks link concept-consistent effects and their reversals in probing and steering to the alignment between concept labels and the answer measure.

Mon 21 SeptComputation and Language
The gist
Language models generate answers by predicting the next word, but how they organize concepts inside is not clear. The authors propose that these models group all possible continuations that yield the same answer into what they call 'answer basins'. These basins capture how likely different answers are, and the shared directions linked to concepts arise from differences in the probabilities of these answer groups rather than explicit concept labels. Their experiments suggest that understanding and influencing concepts depends on how these answer basins align with concept labels.
Open → 2609.24821v1