Summary
Large language models are evolving into smart agents that can plan and use tools, but this creates new challenges for keeping them safe from harmful instructions, called jailbreaking. The authors studied how well old tricks for attacking and defending these models still work as AI grows more powerful. They found that good built-in safety doesn’t guarantee protection against tricky attacks, and defenses can harm usefulness or slow things down. Also, even when the final AI answer seems safe, hidden parts of its thinking or tools it uses might still be compromised. This means future safety efforts need to look deeper into how these AI agents work step-by-step.
What this means in practice
- •For ai safety engineers: Develop multi-layer defenses that monitor and protect agentic AI components beyond just final outputs to improve security.
- •For ai system architects: Design AI agents that balance security, utility, and efficiency by considering vulnerabilities in planning, memory, and tool interactions.
Authors
Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac, M. Hadi Amini
Abstract
Large language models (LLMs) are rapidly evolving from conversational assistants into agentic AI systems that reason, plan, invoke tools, maintain persistent memory, communicate with other agents, and execute multi-step tasks. At the same time, modern models exhibit substantially stronger native safety alignment than earlier generations on which many jailbreak attacks and defenses were originally studied. This shift raises a fundamental question: \textit{which established jailbreak-security findings remain valid in the era of modern LLMs and agentic AI?} We address this question through a Systematization of Knowledge (SoK) that reframes jailbreak security around the full agentic execution pipeline. We develop unified taxonomies of attacks and defenses spanning user interaction, planning and reasoning, memory, tool use, and inter-agent communication, and introduce a security--utility--efficiency evaluation framework that separates native harmful-prompt safety, adversarial jailbreak robustness, and agent-level security outcomes. We further conduct a controlled empirical study of representative attacks and defenses within a common agentic framework. Our results reveal three important gaps. First, strong native alignment does not imply robustness to adversarial jailbreaks. Second, defense effectiveness is highly model-, attack-, and component-dependent and can come at substantial cost in over-refusal, utility, and latency. Third, low final-response attack success can mask severe intermediate compromise: planning, memory, and tool interactions may remain unsafe even when the final response is successfully filtered. These findings motivate a shift from response-centric jailbreak defense toward cross-layer, execution-aware security that protects agent state, component transitions, and external actions while preserving practical utility and efficiency.