High quality data can still introduce safety risks in language models
High-quality Data Do not Mean Safe! Poisoning LLMs after Data Selection
Cryptography and Security
Summary
Large language models (LLMs) can be fine-tuned in ways that make them unsafe, even if the training data looks good. The authors found that selecting high-quality data to train these models doesn't fully block harmful influences. Some harmful samples that appear high-quality can sneak through and still make the model unsafe by affecting its internal learning patterns. They created a method called Bi-QSTO to show these harmful samples can be designed to survive data filtering and still degrade model safety.
What this means in practice
- •For ai safety engineers: Evaluate and improve data filtering methods in training pipelines to detect subtle poisoning threats that evade quality-based selection.
- •For machine learning operations teams: Design monitoring systems that track layer-wise gradient behaviors to flag potentially harmful fine-tuning samples passing quality filters.
Authors
Kaiyang Li, Jiahao Chen, Yuwen Pu, Chunyi Zhou, Tong Zhang, Bin Cai, Chunqiang Hu, Haibo Hu
Abstract
Safety-aligned Large Language Models remain vulnerable to fine-tuning on small sets of harmful or benign-looking samples. However, prior studies typically assume that poisoned samples directly enter downstream fine-tuning, overlooking quality-based selection in practical training pipelines. To fill this gap, we systematically evaluate both the filtering effects against poisoning and the downstream safety impact of retained data. The results reveal that selection removes many overtly harmful samples, yet some retained high-quality samples can still degrade model safety alignment possibly due to their harmful-like training-update patterns at the layer-wise gradient level. Together, these findings expose a practical vulnerability: safety-degrading influence can pass through quality-based selection via retained high-quality samples. To examine its systematic exploitability, we propose Bi-Stage Quality-Constrained Safety-Degradation Text Optimization (Bi-QSTO), which optimizes poisoned samples under an explicit quality constraint to survive selection while preserving their safety-degrading influence. Across poisoning settings, target models, and filtering rates, Bi-QSTO maintains attack effectiveness before and after selection. Even at 90% filtering, harmful-seeded samples achieve a Poisoning Retention Rate above 90% and Harmful Score of 3.30--4.01. Their attack effectiveness strongly transfers across models and their retention advantage generalizes to additional selection methods.