Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate
2026-06-15 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors introduce Tyler, a new method that helps large language models think better by deciding when to do calculations silently (in the background) and when to write out their reasoning step-by-step. Unlike earlier approaches that fixed when and how these background calculations happen, Tyler learns to choose the best moments and types of reasoning needed during the text generation process, making it more efficient. Their tests show that Tyler improves accuracy compared to previous methods and works well across different kinds of reasoning tasks, while also remembering information better over time.
chain-of-thought promptinglarge language modelslatent reasoningautoregressive decodingreasoning functionspolicy learningglobal planningprocedural abstractionaccuracy improvementforgetting
Authors
Hanyu Lin, Min Cai, Jiawei Wen, Haodi Zhang
Abstract
Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose \textbf{Ty}ped \textbf{L}at\textbf{e}nt \textbf{R}easoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.