AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
2026-06-01 • Cryptography and Security
Cryptography and SecurityArtificial IntelligenceComputation and LanguageEmerging Technologies
AI summaryⓘ
The authors study a security issue where smart assistants using large language models (LLMs) interact with third-party tools like Gmail or Jira, but these tools can send back hidden harmful messages the user didn't write. They created a new test called AGENTREDBENCH with many tricky scenarios to better measure how often attacks succeed. They also made a defense model, AGENTREDGUARD, which greatly reduces successful attacks compared to no protection or other open-source guards. Their approach works well across many different tools and attack types, and they share their test and model openly for others to use and improve.
LLM agentsIndirect prompt injectionTool integrationsAuthorization attacksRedteaming benchmarkAttack success rateAdversarial detectionOpen-source guard modelsEnterprise integrationsAGENTREDGUARD
Authors
Hiskias Dingeto, Will Leeney
Abstract
Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls. Existing benchmarks under-measure the threat: most cover only a handful of integrations with the same attack payload replayed across runs, and open-source guards are trained on chat-style data rather than tool-response content. We introduce AGENTREDBENCH, a dynamic LLM-driven redteaming benchmark of 215 subtle underspecified authorization (attacks at the boundary of what the user's request authorises) scenarios across 24 enterprise integrations in nine functional families and five attack types. Across an eight-model panel (Anthropic, OpenAI, Google), no-guard ASR (attack success rate) ranges from 32% (Claude Sonnet 4.6) to 81% (Gemini 3 Flash). To keep the scenario set out of training corpora and preserve headline ASR meaning over time, we release the codebase, integration schemas, and AGENTREDGUARD model openly; the canonical scenarios are evaluated through a maintainer-mediated channel with immutable versioning. We release AGENTREDGUARD alongside the benchmark: a guard trained on an integration-diverse corpus of adversarial tool-response content. AGENTREDGUARD cuts panel ASR from 69.9% to 2.4% at 0.37% false-positive rate, outperforming every open-source baseline with non-trivial detection (Llama Guard, PromptGuard 2, ProtectAI) on both axes. Cross-integration and cross-attack type holdouts both confirm the gain transfers beyond the training subset.