Summary
Some language models can be tricked by special word sequences, called triggers, to change how they respond, but how these triggers work inside the model is not well understood. The authors studied this by training simpler models that learn to spot these triggers and found that different parts of the main model detect triggers, pass along their effect, or actually control the change in language output. They showed that some parts are good at noticing the triggers but do not cause the behavior change, while other parts control it but don’t help detect triggers. This suggests that trigger-based tricks inside language models have separate roles that could apply to other similar backdoors even if they use different words or layers.
large language modelsbackdoorstrigger sequencessparse autoencodersfeature detectionattentionMLP (multi-layer perceptron)residual streammodel ablationlanguage switching
Authors
Wissam Antoun, Francis Kulumba, Théo Lasnier, Benoît Sagot, Djamé Seddah
Abstract
Even though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.