Gated memory in large language models vulnerable to hidden backdoor attacks
BadEngram: Backdoor Attack on Gated Memory Components in LLMs
Cryptography and Security
Summary
Large language models use special memory parts to improve how they work without slowing down too much. The authors found a way to secretly change these memory parts after training to make the model behave badly when it sees certain hidden signals, without changing its main brain or making it obvious. They tested this trick on small and large models, showing it can work well and remain hidden. This means these memory parts are a weak spot that needs careful protection.
What this means in practice
- •For ai security engineers: Detect and secure gated memory parameters to prevent stealthy backdoor implantations in large language models.
- •For enterprise ai deployment teams: Verify gated-memory component integrity separately from backbone weights to maintain safe use of pre-trained models from external sources.
Authors
Ariel Fogel, Omer Hofman, Eilon Cohen, Roman Vainshtein
Abstract
To expand open-weight models' capacity without proportionally increasing computation, recent language models incorporate gated parametric memories that retrieve learned values and inject them into intermediate representations. Despite these efficiency benefits, such modules create a distinct attack surface: their parameters can be modified independently of the backbone while directly shaping its computation. We introduce BadEngram, a post-training attack that exploits this surface to implant persistent, trigger-dependent behavior while leaving conventional backbone weights and the execution graph unchanged. We first establish the attack's feasibility and causally characterize its mechanism in a controlled Engram model, where BadEngram achieves 96.6% ASR on triggered inputs while limiting false activation on matched trigger-free inputs to 0.1% and preserving 99.6% clean accuracy. Replacing the retrieved memory values with their clean counterparts or closing the memory gates reduces ASR to at most 0.32%, confirming that the backdoor is expressed through the gated-memory pathway. We then test whether this vulnerability extends to production scale in Qwen3.8-Flash-Next's native Per-Layer Embedding subsystem. Using independently trained checkpoints for the two benchmarks, BadEngram achieves 50.4% ASR on HarmBench and 60.0% on AdvBench, while dormant-condition ASR remains 0.9% and 0.0%, respectively. These results identify native gated-memory parameters as a security-critical part of the model whose integrity cannot be inferred from an unchanged backbone.