Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
2026-07-20 • Cryptography and Security
Cryptography and SecurityArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors created a benchmark with 21 scenarios to test how well language model (LLM) defenders handle attackers that adapt their strategies over multiple rounds, rather than just one. They found that attacks become more successful when attackers can change based on the defender’s previous responses, compared to single-turn attacks. Using multiple advanced attacker LLMs together discovered more unique attacks than using any one alone. They also showed that different defender models have varying strengths and weaknesses across scenarios, and no single ranking fits all cases. The authors provide this benchmark and related data publicly for future safety testing.
LLM agentsprompt injectionmulti-turn attacksadaptive attacksbenchmarkattack success ratedefenderlanguage model safetyevaluation scenarioscosine similarity
Authors
Devina Jain, David Hartmann, Chuan Li
Abstract
LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to the first attacker turn yields $0$-$1\%$ attack success rate (ASR); allowing 15 rounds of adaptive attack yields $5.4$-$14.0\%$. Pooling three frontier attacker LLMs uncovers $1.4$-$2.2\times$ as many unique successful attacks as the best single attacker, and the generated attacks have low cosine similarity ($0.02$-$0.14$) to attacks in existing benchmarks. Claude Opus 4.6 and GPT-5.4 are tied in aggregate ($5.4\%$ each; overlapping $95\%$ CIs), but their weaknesses differ sharply: on one scenario Opus reaches $60\%$ ASR ($95\%$ CI $36$--$80\%$) while GPT-5.4 and Gemini each stay at $7\%$ (CI $1$-$30\%$; the gap is preserved in a higher-$N$ replication). $13$ of $21$ scenarios distinguish at least one defender pair, yet rankings disagree across scenarios (Kendall's $W = 0.19$). We release the benchmark -- 21 evaluation scenarios, 10 public development scenarios, the orchestrator, baseline harnesses, and a multi-attacker CLI -- plus 945 transcripts from the 3$\times$3 frontier matrix, an attack-replay dataset, and 18{,}422 gpt-oss-20b battles from an open competition's final scoring rounds.