Large language models resist attacks by reshaping internal representations
AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion
Machine LearningArtificial Intelligence
Summary
Some attacks that trick one language model can also fool other very different models, creating a shared security problem. The authors found that these attacks work because models have similar ways of internally representing harmful inputs. They designed a lightweight method called AnchorRep that pushes a model's harmful input representations away from a fixed reference model's, reducing the chance these attacks transfer. This approach works across different model types and does not require training on many attack examples or changing how the model refuses dangerous prompts.
What this means in practice
- •For ai system operators: Deploy this method to reduce risks of attacks designed for one model from compromising other independently deployed language models.
- •For security engineers: Implement lightweight adapters that improve robustness of language models against harmful prompt transfers without degrading normal outputs.
Authors
Gal Wertheizer, Rom Himelstein, Tomer Peretz, Avi Mendelson
Abstract
Adversarial attacks optimized on a single open-weight LLM can transfer to and jailbreak architecturally different models, allowing an attacker with white-box access to one model to compromise independently deployed systems. This creates a shared vulnerability across models, yet existing defenses are not designed for this cross-model threat. We find that cross-model transfer aligns with shared internal representation geometry, making it a natural defense target. AnchorRep targets this geometry directly with a lightweight LoRA adapter that pushes the defended model's internal representations of harmful prompts away from those of a frozen anchor model on the same prompts. Training uses a small set of harmful prompts and no adversarial examples. Across five models and four architectural families, AnchorRep reduces cross-model attack success rate to <=1.1% on 2,000 transferred attacks (0% on two), including the largest drop on Mistral (36% -> 1.1%). Existing defenses can reduce transfer, but only at high cost either inducing up to 77% degenerate benign output or increasing over-refusal by up to 18%. Because such degenerate benign outputs are not captured by standard refusal-based metrics, we introduce the Benign Garble Rate to quantify them. Our results suggest that cross-model robustness can be achieved by shaping representation geometry, without requiring attack-specific training