Dual Alignment Between Language Model Layers and Human Sentence Processing
2026-04-20 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how well different layers of large language models (LLMs) match human brain effort when reading sentences, especially tricky ones with ambiguous grammar. They found that early layers of LLMs match human reading in easy sentences, but for harder sentences, later layers do better at reflecting human struggle, although they still underestimate it. This suggests humans might use simpler predictions for easy reading and more complex understanding for difficult sentences. They also tested new ways to measure reading difficulty that combine strengths from both early and later layers of LLMs.
surprisallarge language modelssentence processingsyntactic ambiguitycognitive effortreading time modelingnaturalistic readingprobability-update measurescontextualized representations
Authors
Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki, Ethan Gotlieb Wilcox
Abstract
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.