Timesteps of Mamba Align with Human Reading Times
2026-06-29 • Computation and Language
Computation and Language
AI summaryⓘ
The authors found that a computer language model named Mamba processes words in a way that matches how long humans take to read each word. In Mamba, the time spent on each word changes dynamically depending on the input. Their study shows that this model's timing predicts human reading speed better than some other known factors. They also explain how Mamba’s layers handle memory over time, which might help us understand human language processing more deeply.
state-space modelMambaper-word processing timedynamic timestephuman reading timesGPT-2 surprisalreal-time language processingmemory retentionnaturalistic reading dataneural network layers
Authors
Yuji Yamamoto, Shinnosuke Isono, Yoshinobu Kawahara, Sho Yokoi
Abstract
This study demonstrates an alignment of per-word processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep $Δ_t$, determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with ever-updated memory, because it allows us to look at how each module (layer) weighs short- and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online.