Model size and compression affect bias and abstention in question answering
Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering
Computation and Language
Summary
Language models can sometimes give biased answers or choose not to answer at all. This paper looks at how making models smaller or compressed (quantized) changes their behavior in answering questions fairly. The authors found that bigger models answer more accurately but aren’t always less biased, and smaller or quantized models might show less bias partly because they avoid answering more often. This means measuring fairness needs to consider both biased answers and when models decide to stay silent.
What this means in practice
- •For ai developers: Develop QA systems with balanced trade-offs between accuracy, bias, and abstention when using smaller or quantized models.
- •For ml model auditors: Design fairness evaluations for QA models that include biased response rates, abstention behavior, prompt wording, and answer formats.
Authors
Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar Thakker
Abstract
The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention? We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2. We also examine whether the observed trends remain stable across alternative prompt formulations and when the ``Unknown'' option is removed in a forced-choice setting. Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations. Notably, 20 of the 26 configurations with fewer biased responses also show increased over-abstention, indicating that lower observed bias can partly reflect greater abstention. These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.