Summary
This paper explores small quantum systems used to predict the next byte in a sequence of text without classical methods like the usual vocabulary projections. The researchers found that adding controlled loss (dissipation) into the quantum memory helps it remember more than one byte at a time, but it still doesn’t outperform classical models using two-byte contexts. They tested these quantum models on real quantum hardware and showed they can generate text, though their accuracy and capacity remain limited compared to classical approaches. The work highlights early steps toward quantum-native language models but does not show a quantum advantage yet.
What this means in practice
- •For quantum software engineers: Prototype language models that run directly on quantum devices for experimentation and benchmarking of quantum memory effects in text prediction.
- •For language model developers: Explore quantum-inspired mechanisms like dissipative memory to inspire new classical architectures with constrained memory dynamics.
Abstract
We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.