Method finds hidden learning signals in datasets used to train AI
Verbalizing Subliminal Learning Effects Using Text Optimization
Machine LearningArtificial IntelligenceComputation and Language
Summary
Sometimes AI models learn secret clues from their training data that are not obvious or written down. This hidden learning can cause risks when training new AI. The authors developed a method called SALVE that can find and explain these secret clues by turning them into understandable text prompts. Their approach helps detect when an AI is learning from hidden signals, even in mixed or biased data. This sheds light on how AI quietly learns and offers a way to watch for potential issues during training.
What this means in practice
- •For machine learning engineers: Detect hidden data influences during model training to improve trustworthiness and control of AI systems.
- •For security teams: Identify and mitigate risks from malicious or biased training data that could manipulate AI behavior.
Authors
Nathan Hu, Sanmi Koyejo, Christopher Potts
Abstract
Subliminal learning is a phenomenon in which a distillation dataset transmits traits from the teacher model that are not legibly encoded in the dataset itself. This introduces a new challenge for model development and creates new risks from data poisoning. In this work, we use text optimization to detect subliminal learning effects and describe them as legible prompts. Subliminal learning from a prompted teacher motivates our approach. We observe that this is a special case of context distillation and leverage this observation to show that, in theory, the prompted subliminal learning dataset identifies the teacher's prompt. We reduce recovering this prompt to a text optimization problem and present a method to approximately solve it. Our method, SALVE (Search-Aided Latent Verbalization), optimizes a soft prompt, queries the same model to verbalize it as text, and uses beam search to make the verbalization reliable. In the standard subliminal learning setting, SALVE reliably recovers legible prompts that name the teacher's trait, while common text optimization methods fail to do so. In addition, we find that there are settings in which SALVE recovers the teacher's trait from a dataset even when subliminal learning fails, but that modifying student training to improve context distillation can create subliminal learning effects. We lastly show that SALVE detects subliminal learning effects in three additional settings: (1) mixtures of subliminal learning data and unrelated data, (2) data generated when the teacher is biased via activation steering, and (3) subsets of real preference data selected via Logit-Linear Selection. Overall, our results deepen our understanding of subliminal learning and present SALVE as a method to proactively detect subliminal learning effects.