Unlearningsoup cuts tuning time for safer large language models
UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
Machine Learning
Summary
Large language models sometimes memorize and repeat harmful or unwanted information. To fix this, people try to make the models "unlearn" certain content, but the ways to do this require lots of trial and error to balance forgetting bad stuff while keeping good knowledge. The authors found that different attempts at unlearning share common ground, so they created UnlearningSoup, a method that mixes these attempts to quickly find better models without repeated searching. This approach saves time and makes unlearning more effective across different cases.
What this means in practice
- •For ai safety engineers: Improve efficiency and effectiveness when removing harmful content from large language models by applying UnlearningSoup rather than repeated tuning.
- •For machine learning operations teams: Reduce hyperparameter tuning resources and speed up deployment of safer language models by using UnlearningSoup during model updates.
Authors
Puning Yang, Qizhou Wang, Junchi Yu, Bo Han, Xiuying Chen
Abstract
Large language models trained on vast corpora inherently risk memorizing harmful content that may later re-emerge in their outputs. To mitigate this issue, existing unlearning methods typically rely on training-based parameter updates, such as gradient ascent and its variants, to delete targeted content while preserving other knowledge. However, balancing the competing goals of forgetting and retention makes hyperparameter choices for these methods particularly difficult, often requiring repeated tuning to obtain a strong model that still leaves substantial room for improvement and transfers poorly across models and datasets. To address this challenge, we investigate whether unlearning runs exhibit exploitable structure in weight space, and observe that models from different runs still lie in a shared evaluation-performance basin. This suggests that stronger models may be recovered through an unlearning-tailored soup strategy, reducing the need for repeated tuning for further improvement or new settings. Motivated by this, we propose UnlearningSoup, a unified framework that provides two strategies: EfficientSoup uses binary-search-based interpolation to quickly discover a well-performing model in the early stage, where repeated tuning would otherwise make strong model selection costly. PerformanceSoup uses reweighted souping to efficiently unlock the remaining performance potential in the later stage, where repeated tuning becomes increasingly inefficient. Extensive experiments across diverse datasets and models show that UnlearningSoup delivers 2.4x to 3.3x efficiency gains in hyperparameter selection, while consistently improving performance across settings.