Persona generalization follows predictable patterns across ai models
Is Persona Generalization Predictable?
Computation and LanguageMachine Learning
Summary
When language models are fine-tuned on one set of behaviors or domains, they can unexpectedly change their behavior in other areas too. The authors found a consistent pattern, called the "on-the-line" phenomenon, showing how these changes relate in a predictable way across different models and sizes. By adjusting which parts of the model are fine-tuned, it is possible to control how much the new behavior generalizes outside the training domain. This helps explain and potentially manage cases where models develop unsafe or misaligned behaviors after fine-tuning.
What this means in practice
- •For machine learning engineers: Design fine-tuning strategies that predictably control model behavior transfer across domains by selecting specific layers to train.
- •For ai safety teams: Reduce unintended unsafe behavior in deployed language models by adjusting fine-tuning approaches to limit out-of-domain misalignment.
Authors
Olina Mukherjee, Arnav Goel, Aditi Raghunathan, Jacob Mitchell Springer
Abstract
Persona generalization occurs when fine-tuning a language model in one domain induces behavioral changes in others -- for example, the broadly unsafe behavior observed in emergent misalignment. We discover a surprising "on-the-line" phenomenon in persona generalization: in-domain and out-of-domain persona rates across various fine-tuning hyperparameters follow an approximately probit-linear relationship, and the specific probit-linear relationship is surprisingly shared even across model families and model sizes, including Qwen and Llama. Training choices that leave the set of trainable parameters unchanged, such as learning rate, weight decay, and training duration, largely move models along the same line, reducing transfer mainly by weakening in-domain learning. Changing which layers are fine-tuned, however, can change the slope. For unsafe behavior, fine-tuning only later layers yields shallower slopes, retaining strong in-domain learning while reducing out-of-domain misalignment. This "on-the-line" phenomenon unifies unexpected, isolated observations of persona generalization within a quantitative framework for predicting and controlling behavioral transfer.