Navigator guided data selection reduces forgetting in model fine tuning
NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning
Machine Learning
Summary
When teaching large language models new tasks, they often forget what they learned before, especially if the original training data isn't available. The authors introduce a method called NADS, which uses a temporary model (navigator) to spot which new data examples are best for training without forgetting old knowledge. This method picks useful and diverse examples efficiently and helps the model keep its previous abilities while learning new ones. Their experiments show better results in balancing new learning and memory retention.
What this means in practice
- •For machine learning engineers: Select effective synthetic data for fine-tuning large language models without losing their general abilities.
- •For natural language processing teams: Improve fine-tuning pipelines by reducing extra computation in choosing training data for new language tasks.
Authors
Jianhao Zhang, Ou Wu, Yi Du
Abstract
Catastrophic forgetting is a central obstacle in single-stage fine-tuning of large language models (LLMs) on a new task, especially when the original pretraining data are unavailable. Synthetic data offers a practical mitigation path, but candidate pools are often noisy and redundant, making effective selection essential. Existing methods often rely on indirect proxy signals whose alignment with actual training benefit can be limited, while utility--diversity selection over large candidate pools remains computationally demanding. To address these challenges, we propose NAvigator-guided Data Selection (NADS), a fine-tuning framework for LLMs. NADS first follows the target fine-tuning recipe without preservation regularization to obtain a navigator model, whose deviation from the pretrained model exposes recipe-induced capability drift. It then uses the predictive divergence between the navigator and the pretrained model to score forgetting-aware utility for each synthetic candidate, and constructs a constraint set through an efficient utility--diversity selection strategy. Finally, NADS fine-tunes on the new-task data while distilling pretrained behavior on the selected constraints to preserve general capabilities. Experiments show that NADS delivers a stronger balance between new-task performance and general capability preservation, while reducing utility--diversity selection overhead. Code at: https://github.com/zjh-vinky/NADS.