Papers for

bias mitigation teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Large language models often see false patterns more than humans do

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

Abstract: Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tendency, often described as "connecting the dots" where none exist, can result in systematic reasoning errors. This paper investigates whether Large Language Models (LLMs) exhibit such perceptual tendencies, which can lead to systematic errors in downstream applications. To our knowledge, this work presents the first systematic study of illusory pattern perception in LLMs, adapting classic psychological paradigms to three tasks with direct empirical comparison to human behaviors. We find that LLMs frequently exhibit stronger illusory pattern perception than humans. In particular, models tend to over-associate frequent positive attributes with majority groups or large organizations, and show increased tendencies to construct causal narratives from ambiguous events. To uncover the mechanism behind these behaviors, we develop a feature interpretability framework based on Sparse Autoencoders (SAEs) to analyze internal representations. Our results reveal that holistic frequency perception and analytic cognitive orientation are linked to the emergence of illusory perceptions. These findings highlight a previously underexplored cognitive-like illusion that may affect the reliability of LLM reasoning. Code available at https://github.com/NusIoraPrivacy/illusory.

Tue 6 OctArtificial Intelligence
The gist
Humans sometimes mistakenly find meaningful connections in random information, called illusory pattern perception. The authors studied whether large language models (LLMs) have the same tendency and found that these models can actually see false patterns more often than people. For example, LLMs might wrongly link popular positive traits with big groups or invent causes from unclear events. The authors used a special method to peek inside the models and found features that explain why these mistakes happen. This suggests that LLM reasoning can sometimes be less reliable than expected.
Open → 2610.07791v1

Large language models unevenly recognize names by race and gender

Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

Abstract: Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such evaluations typically assume that matched names are comparable model inputs. We show that this assumption often fails at the lexical interface: matched names are not necessarily matched inputs. Some names receive direct single-token access, while others are assembled from multiple subwords, creating unequal name-surface support. Across nearly half a million first names and 12 LLM-associated tokenizers, direct lexical access is highly selective, model dependent, and uneven across race- and gender-associated name metadata. We introduce NameTrace, a model-native, fine-grained, pre-behavioral framework for measuring whether unequal name-surface support remains a vocabulary property or becomes visible in task-relevant internal representations. NameTrace measures concept accessibility from the model's own probabilities over task-specific adjective axes with continuous task-aligned weights. On matched atomic and short-fragmented names within the same race/ethnicity--gender-associated strata, support predicts systematic differences in concept accessibility across fellowship, hiring, clinical assessment, and lending. These differences persist across all eight matched strata, extend across model families, and transfer to unseen names. Hidden-state interventions further show that the measured task directions have downstream leverage, shifting later constrained choices. Unequal lexical support is therefore demographically structured at the input and remains visible in task-relevant model computation. NameTrace makes lexical comparability measurable, supporting a broader principle: behavioral comparability begins with lexical comparability.

Mon 28 SeptComputation and LanguageArtificial IntelligenceComputers and Society
The gist
Different names aren't always treated equally by large language models because some names are recognized as single tokens, while others are broken into smaller parts. The authors show this unequal tokenization happens more often depending on the name's race or gender associations. They created a method called NameTrace to measure how this affects models’ understanding of concepts linked to those names. Results indicate that these differences matter for tasks like hiring or lending decisions made by the models.
Open → 2609.34065v1