Active adaptation improves defense in harmful AI fine-tuning
Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning
Computation and LanguageArtificial IntelligenceCryptography and SecurityMachine Learning
Summary
Large language models can be tricked by bad updates that make them behave harmfully. The authors studied a way to protect these models by actively adjusting their training signals rather than relying on fixed changes. They found that this active process is what keeps the model safe over time. To improve this, they suggest gradually increasing the strength of these protective signals during training, which better reduces harmful behavior in tested models.
What this means in practice
- •For machine learning engineers: Enhance AI model safety by applying progressive strength scheduling to fine-tuning defenses against harmful persona shifts.
- •For ai safety teams: Maintain robust preventative steering in language models by implementing active adaptation strategies instead of static protective measures.
Authors
Jing Guan, Yachao Yang, Zhaoliang Liu, Yuyao Zhang, Fanyu Meng, Junlan Feng
Abstract
Large language models remain fragile against malicious fine-tuning, motivating training-time defenses against harmful persona drift. Preventative Steering injects undesirable-trait persona vectors during fine-tuning and removes them at evaluation time, yet the mechanism behind its lasting protection remains unclear. Analyzing its temporal optimization dynamics, we find that the defense emerges from an early compensatory adaptation phase followed by a steady-state phase where the corrective signal decays; in parameter space, attention output projections emerge as the dominant residual-write route for defensive updates. Through Intervention Delta Preservation (IDP) and IDP Continuation experiments, we further show that preserving or reinjecting the weight offset fails to maintain protection, indicating that preventative steering relies on active adaptation rather than a static defense. Motivated by this finding, we propose Progressive Intensity Scheduling (PIS), which starts with a moderate injection strength and increases it after static-strength alignment begins to decay. Across the evaluated Qwen2.5 and Gemma-3 models, PIS improves safety robustness over static-strength steering while reducing harmful trait expression.