How Much is Left? LLMs Linearly Encode Their Remaining Output Length
2026-07-06 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors studied how large language models seem to have a sense of how long their responses will be even before they start generating words. By analyzing hidden internal states of several models, they found that the total response length can be predicted early on, and this prediction works across different types of text. They also noticed that when a model changes its mind and revises a response, its internal estimate of remaining length goes up, showing the model might keep a rough plan of how much it will say. This suggests that language models internally track response length in a way that can be detected but might not directly control output.
large language modelshidden statesresponse length predictionlinear probestoken generationtransformersinternal representationsmodel interpretabilitycompletion tasksplanning in AI
Authors
Mohamed Amine Merzouk, Dmitri Carpov, Mirko Bronzi, Damiano Fornasiere, Adam Oberman
Abstract
Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts. We ask whether the model carries an internal estimate of how much response remains. Training minimal-capacity linear probes on frozen hidden states of three open-weight 7-8B models across seven completion-style datasets, we find three converging pieces of evidence. First, total response length is linearly decodable from the prompt's last hidden state alone, before any output is emitted. Second, probe directions trained on natural-language datasets transfer broadly, including to controlled synthetic completions never seen in training, outperforming a statistical baseline; the converse direction generally fails, and this asymmetry is itself informative. Third, on curated high-loss completions, the probe's per-position estimate shifts upward at the moment the model retracts and restarts a partial solution, a directional behavior no position-only predictor can reproduce (qualitative, not aggregate). We frame this as approximate estimation of remaining generation length, distinct from exact-counting impossibility results for transformers, and interpret it as evidence that LLMs maintain a plan-like internal representation of output length (decodable, not necessarily used causally).