Model fine-tuning can strengthen prior behavior in AI systems
Alignment Inertia: Auditing the Durability of Training Data Influence Through Policy Override Resistance
Artificial Intelligence
Summary
When people try to change the behavior of AI models by giving new instructions or training on new data, it’s not always clear if these changes really work. The authors studied how often such changes actually override what the AI has learned before. They found that some models keep behaving like they did originally, even after trying to change them, a phenomenon called alignment inertia. They also created a way to predict when changes will fail, helping operators understand when their interventions might not work as expected.
What this means in practice
- •For ai platform operators: Measure when system prompts or fine-tuning fail to override existing model behavior to improve trust and control.
- •For content moderation teams: Predict resistances in AI moderation models to better design interventions reducing misinformation and hate speech.
Authors
Renata Barreto, Markelle Roesti, Mohammad Tahaei
Abstract
Platform operators increasingly rely on system prompts and fine-tuning to govern model behavior, yet it remains unclear how reliably these interventions override behavior inherited from prior training. We propose Override Success Rate (OSR) and alignment inertia to measure when operator interventions succeed or fail to change prior behavior. We evaluate zero-shot prompting and LoRA fine-tuning across Llama and Mistral in medical misinformation and hate speech. Alignment inertia persists across both models but varies by model, domain, and policy direction. Notably, in Mistral's restrictive hate-speech condition, LoRA increased inertia by 46.5 percentage points, showing that fine-tuning can reinforce rather than override prior behavior. We also use TRAK to test whether inertia is associated with weaker adaptation signals. TRAK achieves AUC of at least 0.85 in 7 of 8 conditions and outperforms model confidence, TF-IDF similarity, and embedding similarity as a predictor of inertia. These results provide an operator-facing audit of where prior training constrains downstream model governance.