Networked generative models create complex self-learning loops
Stability and Diversity of Networked Self-Consuming Generative Ecosystems
Machine LearningComputers and Society
Summary
It is becoming harder to tell computer-made content from real data because AI models often learn from their own creations. The authors study what happens when many AI models learn from each other’s generated content instead of just their own. They use a network approach to map these interactions and find out when the group’s learning stabilizes or stays diverse. This helps understand how sharing synthetic data affects the behavior and reliability of multiple AI systems working together.
What this means in practice
- •For machine learning engineers: Manage and design multi-model training pipelines more robustly by predicting when collective retraining will stabilize or degrade.
- •For data platform teams: Control data flows across synthetic datasets to maintain diversity and quality in AI model updates from multiple sources.
A theory result. No direct application yet.
Authors
Xiukun Wei, Yang Zhang, Xueru Zhang
Abstract
The widespread deployment of generative AI has made it increasingly difficult to distinguish synthetic content from real data. Consequently, synthetic data is inevitably incorporated into the training pipelines of future model generations, forming a self-consuming training loop. Prior work has studied the effects of such recursive self-consuming training, but analyses have largely been limited to isolated models, where a model consumes only its own synthetic data, or to simplified interactions between two models. This paper takes a first step toward understanding networked self-consuming generative models, in which multiple models consume synthetic data generated by one another through complex interaction pathways. We introduce a theoretical framework representing models as nodes in a directed, weighted graph, with edge weights governing the flow of synthetic data among models. Using this framework, we analyze the long-term behavior of networked models under retraining dynamics, establishing conditions for convergence and characterizing the resulting fixed points. We further investigate how the system's long-term stability and diversity are shaped by each model's access to real data, cross-model data consumption, and the structure of the interaction graph.