Data selection method improves language model training and transferability
Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Training large language models requires picking the best data from huge, mixed collections. The authors found that using a network to select data directly often leads to unstable training and biases toward easy-to-learn examples. They created a new approach called TESS that uses a different way to match data scores, which helps models learn better and generalize across different datasets and model sizes. This method worked well for improving model safety and specific instruction tuning tasks.
What this means in practice
- •For machine learning engineers: Improve training data filtering to boost safety and instruction-following quality in large language models.
- •For ai system integrators: Scale language model training by selecting relevant data subsets transferable across datasets and model sizes.
Authors
Zilin Du, Bowen Yang, Boyang Albert Li
Abstract
Data selection is critical for training large language models on massive and heterogeneous corpora. Meta-learning for Training-data Selection offers a principled alternative to heuristic scoring by learning data weights from a target validation objective, but existing methods face a trade-off between fine-grained valuation and transferability to unseen data. A natural solution is to replace per-sample weights with a selection network. However, we find that directly incorporating such a network into existing MTS objectives leads to unstable optimization and poor generalization, caused by weight suppression and persistent reliance on easy-to-learn features. To address these issues, we propose Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching objective (PVM). Experiments on LLM safety and targeted instruction tuning demonstrate strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.