Model fragility varies widely in recursive training collapse scenarios
A Fragility Spectrum for Recursive Language-Model Training
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Sometimes AI language models train on text they or similar models generated before, which can make their future text less varied and repetitive. The authors show that some models are very fragile and quickly fall into repetitive patterns, while others barely change, even when trained the same way repeatedly. This difference depends on the specific model checkpoint and can’t be predicted by usual measures like size. However, by letting a model generate text for a few rounds, you can tell if it’s fragile. Adjusting how the model generates text or filtering the data can slow or almost stop this collapse.
What this means in practice
- •For machine learning engineers: Identify and select robust language model checkpoints that resist repetitive collapse during retraining on mixed human and generated text.
- •For content moderation teams: Apply data filtering and generation controls like tightened top-p to maintain diversity and quality in datasets containing AI-generated text.
Authors
Yangze Liu, Zhongyi Han
Abstract
Model-generated text is finding its way back into training corpora, and there is plenty of evidence that training on such data over and over collapses output diversity. Prior work has studied the phenomenon itself: which protocols and which data mixtures cause collapse. But different models behave very differently under the same process. We fix one recursive contamination protocol and let 13 publicly released checkpoints form an ecosystem that shares a common corpus for five generations. The unique 4-gram outcome after five generations ranges from 0.187 to 0.940 across checkpoints, a roughly five-fold spread: some models are barely touched, others degenerate into repetitive fragments. Changing the composition of the shared pool or mixing in human text keeps the Spearman correlation of the ordering at 0.91--0.97, and changing the random seed keeps it at 0.93--0.98. Whether a model collapses easily under recursive training is, then, a property of the checkpoint itself, and one that has gone largely unexamined. Parameter scale alone does not explain it, since a three-size ladder within one family is not monotonic in size, and none of the static indicators we tested predicts it either. What does work is cheap: let a model iterate on its own output for two or three generations, and its fragility in the larger ecosystem can be inferred from that alone. Collapse speed also responds to intervention. Tightening top-p, which cuts the low-probability tail at generation time, nearly stops collapse within three generations and stabilizes six checkpoints spanning the whole spectrum together, while data-side filtering slows collapse without stopping it.