Synthetic data contamination detection improves language model training safety
SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data
Computation and Language
Summary
When large language models train using data that includes artificial text from other models, the resulting model can lose accuracy even if it sounds fluent. The authors identify a way to detect such synthetic data in training sets before using it, without needing to know how the fake text was generated. They measure unusual patterns in word diversity, phrase endings, and prediction difficulty to score datasets for contamination. Their tool, SynthSentry, can help data curators filter out problematic text and avoid training issues caused by synthetic data.
What this means in practice
- •For language model developers: Screen training datasets for synthetic text contamination before using them to prevent accuracy and quality degradation in new language models.
- •For data curation teams: Use a model-agnostic scoring tool to rank and filter text corpora by contamination severity without needing access to generation history or synthetic labels.
Authors
Praveen Kumar Myakala, Ravichandra Namburi, Sowmya Keragodu Jayaramu, Sooraj George Thomas
Abstract
Large language models trained recursively on their own or other models' outputs undergo model collapse, in which distributional tails and factual accuracy deteriorate while fluency survives. Prior work diagnoses collapse after training; the actionable problem is screening a corpus of unknown provenance before training. We introduce SynthSentry, a corpus-level, model-agnostic contamination signal requiring no access to the generating model, no generation history, and no synthetic labels. The score is a distributional divergence over three statistics: lexical diversity collapse, n-gram tail truncation, and perplexity variance across reference models. We evaluate on corpora contaminated by small open-weight generators and an instruction-tuned open-weight model under a leave-one-generator-out protocol. A domain-stratified study measures false positives on naturally repetitive human text (legal, clinical, source code). The score ranks corpora by severity with little loss when whole generator families are held out. Per-domain calibration holds near its nominal false-positive budget once covariance shrinkage and a bootstrap threshold replace a naive quantile, which runs four times over budget. A downstream fine-tuning check showed no contamination-driven accuracy deficit at our scale, so whether pruning recovers one remains open; the same run shows over-pruning risk once pruning exceeds the true contamination fraction. We frame screening as a data-curation defense rather than a post-hoc diagnosis and release the scoring toolkit. All results are small-scale; scope is English-language, batch-mode corpus screening. Contamination sources are single-generation or hand-authored rather than recursively generated, so results speak to synthetic contamination generally and not to recursion depth.