Looped transformers reduce resources for complex computations
Internalizing Chain-of-thought into Looped Transformers
Computational ComplexityData Structures and AlgorithmsMachine Learning
Summary
Transformers are computer models that process information by generating and using tokens. Normally, transformers use many tokens to handle large computations, which can be resource-heavy. This paper shows how looping the transformers’ internal steps can cut down the tokens and memory needed to simulate complex computations. The authors create a method that balances generating tokens and using hidden states, so the model runs more efficiently while still handling complex tasks.
What this means in practice
- •For machine learning engineers: Design transformer models that use fewer memory and token resources for complex sequential tasks.
- •For embedded systems developers: Develop efficient AI systems capable of reasoning with limited hardware memory and token generation capacity.
A theory result. No direct application yet.
Authors
Kehang Zhou, Pinyan Lu
Abstract
Chain-of-thought (CoT) enables constant-bit-size transformers to perform general-purpose computation through generated tokens (Li and Wang, 2025a), whereas looped transformers carry out the same computation by recurrence in hidden states (latent thought). We construct a family of constant-bit-size looped transformers that interpolates between CoT and latent thought. For a prescribed loop count $L=L(n)$ and a fixed integer $k\ge1$ satisfying $s(n)^{1/k}=Ω(L)$, our transformer simulates a time-$t(n)$, space-$s(n)$ Turing machine using a context window of $O(s(n)/L)$ and $O(t(n)s(n)^{1/k}/L)$ generated tokens, reducing both resources by a factor of $L$ relative to the corresponding unlooped queue-machine simulation. The construction also holds with softmax attention at constant bit-size, under the finite-precision kernel specified in this paper.