A Model with No Head and Many Thoughts
2026-08-31 • Machine Learning
Machine LearningComputation and Language
AI summaryⓘ
The authors present Soft Latent Thinking, a new method that improves how large language models generate reasoning steps. Instead of converting thoughts into words at each step, their method keeps reasoning in a continuous space, which is faster and less computationally expensive. Tests show that their approach improves accuracy while reducing the amount of computing needed. This suggests that language models can think effectively without always having to turn their thoughts into tokens.
large language modelsautoregressive decodingembedding spacechain-of-thought reasoningtokenizationcompute efficiencylatent representationspass@k metric
Authors
Nikita Koriagin, Yaroslav Aksenov, George Bredis, Gleb Gerasimov, Nikita Balagansky, Daniil Gavrilov
Abstract
Large language models decode by projecting hidden states through a large vocabulary head at every step. This operation is computationally costly and forces all reasoning to be expressed in discrete tokens. We introduce Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector, enabling autoregressive rollout in embedding space where reasoning steps remain continuous rather than tokenized. Experiments on DeepSeek-Qwen-1.5B and LLaMA-3.2-3B show that Soft Latent Thinking consistently improves pass@k across all k while reducing per-step compute during chain-of-thought. Our method achieves the highest pass@32 among all soft-thinking approaches, demonstrating that effective reasoning can be carried out in continuous space without discrete token generation.