Papers for

ml model auditors

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Model size and compression affect bias and abstention in question answering

Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering

Abstract: The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention? We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2. We also examine whether the observed trends remain stable across alternative prompt formulations and when the ``Unknown'' option is removed in a forced-choice setting. Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations. Notably, 20 of the 26 configurations with fewer biased responses also show increased over-abstention, indicating that lower observed bias can partly reflect greater abstention. These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.

Wed 21 OctComputation and Language
The gist
Language models can sometimes give biased answers or choose not to answer at all. This paper looks at how making models smaller or compressed (quantized) changes their behavior in answering questions fairly. The authors found that bigger models answer more accurately but aren’t always less biased, and smaller or quantized models might show less bias partly because they avoid answering more often. This means measuring fairness needs to consider both biased answers and when models decide to stay silent.
Open → 2610.29657v1

Language model outputs traced to key internal components

Matryoshka attribution: Learning to attribute language model outputs to representations and weights

Abstract: Attributing language model outputs to their internal computations is an open problem in interpretability. Existing methods, which use causal interventions, gradients, or learnable masks, either are infeasibly expensive or struggle to identify actual causally-important internal computations. We propose framing attribution as the problem of identifying nested subsets of internal components which minimise a downstream loss. To learn this task, we introduce Matryoshka Attribution (MAttr), a mask learning method that parametrises the mask with a simple differentiable sigmoid top-$k$ operator. We supervise training over all sparsities simultaneously by randomising $k$ over training, resulting in a learned ordering of components by attribution score. MAttr achieves number 1 on the official leaderboard of the Mechanistic Interpretability Benchmark (Mueller et al., 2025); our method identifies sparse and task-transferrable circuits across varying circuit bases. As a practical application, we show that MAttr can be trained with reinforcement learning to identify weight changes responsible for downstream behaviours in LLM finetuning. We train MAttr on refusal judge scores and find that restoring $1\%$ of Llama 3.1 8B Instruct's weights to their base model state is sufficient to remove refusals while maintaining capabilities. We view MAttr as a successful formulation of interpretability into a learnable objective that we can tackle with gradient descent, and encourage future work along these lines.

Tue 22 SeptComputation and LanguageMachine Learning
The gist
It is hard to figure out which parts inside large language models actually make their answers. The authors propose a new way called Matryoshka Attribution that learns to find nested groups of important components by minimizing errors. Their method can rank parts by how much they contribute and found small groups that explain model behavior well. They also showed it can identify the weight changes causing an updated model to refuse certain requests, while keeping other abilities intact.
Open → 2609.25518v1