Summary
When using synthetic data generated from models trained on earlier synthetic data, a process called recursive synthetic-data training, models can degrade in quality. The authors study how this process can also cause errors in the earlier step of choosing important variables, where variables are ranked against ever-changing synthetic labels instead of the true data labels. This mismatch can cause good variables to be discarded—a problem the authors call screening collapse. They analyze this problem mathematically and propose methods to reduce it, showing that keeping some real data or accumulating more data can help avoid this issue. Their experiments on gene expression data support these conclusions.
What this means in practice
- •For data scientists: Improve feature selection accuracy in models trained on recursively generated synthetic data by mitigating variable screening errors.
- •For bioinformatics teams: Enhance gene-expression data analysis pipelines by understanding and addressing variable screening collapse caused by synthetic label drift.
Tested on one dataset.
Abstract
Recursive synthetic-data training can degrade a fitted model, but its effects on the variable-screening stage that precedes model fitting are less understood. When each generation labels data for the next, a marginal screener ranks variables against the current pseudo-label target rather than the original response. Across generations, this target drift can erode or reverse the active--inactive score margin and cause screening to discard variables that clean-label screening would retain. We call this process \emph{screening collapse}. In a Gaussian linear model with minimum-norm recursive generators, we establish finite-sample screening bounds, characterize a late-generation information-loss limit, and derive a minimax impossibility result. Screening collapse can precede and compound ordinary model collapse: omitted variables incur a covariance-weighted loss that no refit restricted to the selected variables can eliminate, while recursive labels further degrade estimation on the retained variables. We also examine two approaches to mitigating screening collapse. Fixed-fraction real-label retention preserves a clean-label component but leaves resolution fixed, whereas data accumulation can avoid screening collapse if the pooled target preserves a positive margin and the growing sample resolves it. Numerical experiments on gene-expression data support the theoretical findings.