Persistent tool poisoning attacks undermine llm agents across conversations
SEAL: Towards Persistent Tool Poisoning Attack against LLM Agents via Prefix-Eliciting Trigger
Cryptography and Security
Summary
Large language model (LLM) agents use tools to help answer questions or complete tasks, but attackers can trick these agents by hiding harmful instructions inside tool descriptions. Previous attacks only worked well in one-time interactions and stopped working over multiple conversation rounds. The authors found that the agent's changing reasoning made these attacks weaker over time. They created SEAL, a two-step method that makes these attacks last over many conversation turns by crafting special triggers that keep the attack active. Their tests showed SEAL is effective, works on different models, and is hard to defend against.
What this means in practice
- •For security teams: Evaluate robustness of LLM-based agents against persistent tool poisoning attacks in multi-round interactions.
- •For developer operations teams: Design monitoring strategies to detect and mitigate adversarial triggers that persistently manipulate agent tool usage.
Authors
Nanjun Zhou, Yiming Li, Yukun Chen, Ting Qiao, Enhao Huang, Zhan Qin
Abstract
Tool poisoning has emerged as a critical security threat to LLM agents, where adversaries embed malicious instructions in tool descriptions or external resources that enter the agent's context, allowing adversaries to manipulate agent's tool selection process. However, existing tool poisoning attacks primarily rely on an idealized assumption that the attacks task effect within a single interaction round. We show that their effectiveness degrades sharply in realistic multi-round agentic settings. We further reveal that this degradation stems from the fact that agents do not consistently surface attack-relevant signals in their reasoning as the planning context evolves, thereby weakening the attack's influence across rounds. To investigate whether tool poisoning remains a practical threat in such realistic settings, we propose SEAL, a two-stage method for persistent tool poisoning. Specifically, SEAL first performs Prefix-Eliciting Trigger Optimization (PETO) to optimize context-specific triggers that elicit attack-relevant reasoning prefixes, improving effectiveness within individual contexts. It then applies Local-to-Universal Trigger Aggregation (L2U) to combine these local triggers into a single universal trigger, improving generalization across different contexts and execution rounds. Extensive experiments across seven representative multi-round scenarios and different models demonstrate the attack effectiveness of SEAL, its transferability across target models, and its resistance to potential defenses.