Llm email agents fail to detect and resist adaptive phishing attacks

AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack

Cryptography and SecurityArtificial Intelligence

Summary

Phishing is a trick to get people or computers to reveal private information or do something harmful. The authors created a new test called AgentPhishing that checks if smart email assistants using AI can be fooled by phishing tricks that are specially designed to fool them. They found that some assistants can easily be tricked and will follow harmful requests even if they detect the phishing. The study shows that old tests for phishing don’t catch these new sneaky attacks and it is important to test how these AI agents behave when they get tricky phishing emails.

What this means in practice

  • For email security teams: Evaluate how well AI-based email assistants handle sophisticated phishing emails tailored to evade detection and induce harmful actions.
  • For enterprise cybersecurity teams: Design improved phishing defenses by understanding AI agent vulnerabilities to persuasive phishing attacks identified by AgentPhishing.

Authors

Justin Tan, Miao Li, Ding Zhao

Abstract

Computer-use agents increasingly handle private information and act on users' behalf across desktop and web applications, including email. Phishing therefore poses privacy and security risks that depend not only on whether agents recognize malicious messages, but also on how they respond. Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized. To expose vulnerabilities that pre-existing datasets may miss, we introduce AgentPhishing, a benchmark for evaluating whether persuasive phishing attacks can evade detection and induce LLM email agents to comply with malicious requests. We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint. We analyze the generated emails using Cialdini's persuasion principles to identify strategies associated with agent failures and use these findings to construct a universal generation skill. Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10\%$ and malicious-request compliance rates above $45\%$. These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.