Gradient methods inconsistently identify data causing hidden model behaviors
Can Data Attribution Filter Out Subliminal Learning? Not Reliably
Artificial Intelligence
Summary
Sometimes language models learn hidden behaviors from training data that doesn’t seem related to those behaviors, making it hard to stop with usual filters. The authors studied special gradient-based methods that try to find exactly which parts of the training data cause these hidden behaviors. They found that some methods work better in certain cases, but overall these techniques are not reliably consistent. This means it’s still tough to filter out unwanted hidden learning just by tracing back to the training data.
What this means in practice
- •For machine learning engineers: Use gradient-based attribution to partially identify training data causing undesired model behaviors when semantic filters fail.
- •For ai safety teams: Implement token-level filtering with EK-FAC gradients to mitigate some subliminal learning risks in deployed language models.
Authors
Moritz Weckbecker, Sweta Jena, Jonas Müller, Ponnurangam Kumaraguru, Sebastian Lapuschkin, Wojciech Samek, Louis Jaburi, Gonçalo Paulo
Abstract
Subliminal learning allows language models to transmit behavioral traits through training data with no obvious semantic relationship to those traits, undermining content-based data filtering as a safety intervention. Training data attribution offers an alternative: it identifies the training examples responsible for a given model behavior, independent of their semantic content, and so may apply in exactly the cases where semantic inspection fails. We evaluate three gradient-based attribution methods (GradCos, a contrastive GradCos variant, and EK-FAC) across three models, comparing them against divergence tokens, a strong baseline previously shown to localize subliminal learning (albeit one that requires access to counterfactual teacher models). Filtering at the token level, EK-FAC mitigates a significant part of the effect, the other methods provide little benefit, and all mostly fall short of divergence tokens. Filtering entire samples is less effective for every method, though EK-FAC often gives a stronger signal than divergence tokens in this setting. Success is inconsistent across methods and settings: variants that work well for some model-preference combinations fail for others, and we do not identify a consistent explanation for these differences. Our results suggest that gradient-based attribution can identify data responsible for subliminal learning in some settings, but that some approximations are more reliable than others.