New method protects language models from harmful fine-tuning attacks
SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
Cryptography and SecurityArtificial Intelligence
Summary
Fine-tuning lets people customize large language models for special tasks, but bad actors can fine-tune them to ignore safety rules while still working well. The authors found that changing different model layers can either fix or worsen safety issues. They built SLDR, a defense that repairs only the most affected layers and smartly decides when to apply this fix. This method greatly reduces harmful responses from the model without hurting its normal task performance.
What this means in practice
- •For language model deployers: Protect deployed language models from harmful outputs caused by malicious fine-tuning with a lightweight post-process fix.
- •For ai safety engineers: Enhance safety monitoring tools by focusing on layers with strongest safety impact and selectively fixing them.
Authors
Hui Zhang, Yachao Yuan, Jiayun Wang, Yuanzhuo Li, Hongtao Wang, Yali Yuan
Abstract
Fine-tuning-as-a-service enables users to adapt aligned large language models (LLMs) to specialized tasks, but malicious fine-tuning can erode refusal behavior while preserving task performance on legitimate inputs. We revisit recent layer-wise safety diagnostics and find that safety sensitivity is signed: scaling different layers can strengthen refusal, weaken it, or have little effect. Motivated by this observation, we propose SLDR, a post-fine-tuning defense based on Selective Layers Recovery and Dynamic Routing. SLDR trains a LoRA recovery adapter only on the layers with the maximum and minimum sensitivity scores in the signed spectrum, and uses representation-based dynamic routing inference to activate the adapter only for malicious queries. Across four model architectures, five downstream tasks, and four harmful benchmarks, SLDR substantially reduces harmful outputs while preserving downstream utility. On Llama3.1/SST2, SLDR reduces the average harmful score from 11.54 to 0.08 while maintaining downstream accuracy, and the harmful score remains near zero under poisoning ratios up to 0.9. The code is available at https://github.com/Stardust457/SLDR.