Quantum models near critical points show scaling like language models
Do Quantum Models Scale Like LLMs?
Machine Learning
Summary
This paper looks at how a quantum model called RydbergGPT learns from data collected by measuring a quantum system made of Rydberg atoms. The authors find that when the quantum system is near a special state called the critical point, the model’s learning improves in a predictable way similar to large language models. Away from this critical point, the learning pattern breaks down. They also compare quantum data to natural language data and find similarities near the critical point, suggesting that the complexity of data influences how well models scale.
What this means in practice
- •For quantum computing engineers: Optimize quantum data-driven models by tuning parameters near critical points to improve training efficiency following known scaling laws.
- •For machine learning practitioners: Design training datasets that capture multi-scale dependencies to achieve more stable scaling behavior in transformer models.
Authors
David S. Berman, Ying-Jer Kao, Roger G. Melko, Alexander G. Stapleton
Abstract
In this work, we study the neural scaling laws of RydbergGPT, an autoregressive transformer model trained on qubit projective measurement data gathered from interacting Rydberg atom arrays. The quantum system is known to exhibit a finite-size remnant of a critical point as the laser detuning parameter is varied. We find that near the critical point the transformer loss as a function of training dataset size is well described by a power-law with a loss floor correction. However, away from criticality the quality of the power-law description is substantially reduced. We then compare the statistical structure of both Rydberg measurements and natural-language corpora using an entropy-normalised, finite sample corrected mutual information "two-point" function. We find that near-critical statistics of the two point functions are closest to those observed in natural-language, whilst other qubit configurations far from the critical point have two-point functions that decay more rapidly. This supports the hypothesis that multi-scale dependence contributes to stable neural scaling, and that scaling behaviour should be viewed as a property of the model-data pair.