Word frequency patterns follow thermal energy and condensation laws
Thermodynamics of words
Computation and Language
Summary
This paper shows that the way words appear in books can be compared to how energy spreads out in physics. Different words act like little oscillators sharing energy, and their frequencies follow a pattern very similar to a known thermal process called Rayleigh-Jeans condensation. By analyzing famous texts like the Bible and Shakespeare's works, the authors find that many rare words cluster similarly to particles condensing at low energies in physical systems. This surprising connection also relates to how wealth distribution behaves in economics.
What this means in practice
- •For language data analysts: Use thermalization models to better characterize and predict word occurrence patterns in large text corpora.
- •For natural language processing developers: Incorporate thermodynamics-inspired frequency distributions to improve statistical language models and rare word handling.
Authors
Klaus M. Frahm, Dima L. Shepelyansky
Abstract
We introduce the word thermalization hypothesis according to which in a literary work there are effective interactions between words that drive them to thermalization where their word frequency distribution is the same as for the phenomenon of Rayleigh-Jeans (RJ) thermalization and condensation. In this concept, the different words play a role of classical oscillators (particles) and their word frequencies, or related tokens, are analogous to certain filling factors over energy state levels. The thermalization process is characterized by the conservation of total system energy, or total number of words/tokens, and total probability norm, or total number of different words. We apply this approach to analyze 16 literary works, including Bible, complete works of Shakespeare, and data from American National Corpus. We show that for these cases the effective system energy is very low and thus there is the RJ condensation phenomenon with a big fraction of total probability concentrated at the lowest energy states corresponding to a big fraction of rare words appearing only once or twice in the whole text. Such a RJ condensation is observed for light propagation in multimode optical fibers and other physical systems. We relate the thermalization of words to the recently proposed wealth thermalization for households in world countries. The economy concepts of Lorenz and Pareto curves, and also Zipf curves, are applied to the word frequency distributions of the selected 16 cases and it is shown that the thermalization theory gives a strikingly good description of real data.