High scoring AI models are not always best for further training

Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack

Artificial IntelligenceComputation and Language

Summary

People usually pick AI models that do best on tests or training to improve them further. The researchers show that this approach can fail when working with very large AI models called mixture-of-experts. Some models that don't score highest initially actually turn out better after more training because they handle small changes well. The team found that models with what they call higher 'solution density' tend to keep their good performance even if their settings are tweaked a bit.

language modelcheckpointpretrainingdownstream trainingmixture-of-expertsloss functionsolution densityweight perturbationsbenchmark score

Authors

Sohir Maskey, Philipp Scholl, Jonas Knupp, Pit Neitemeier, Sascha Wirges

Abstract

Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the best starting point for subsequent training. We show that this assumption can fail in a full 30B mixture-of-experts training pipeline. The checkpoints that perform better after the full downstream training stack also have higher solution density, i.e., retain downstream performance under local weight perturbations.