Security risks in autonomous ai systems need more thorough testing

Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges

Cryptography and SecurityArtificial Intelligence

Summary

Agentic AI refers to smart systems that do more than just create text—they keep information over time, take actions, use tools, and interact with people or other AI systems. The authors found that existing studies often look at only one aspect of these systems’ security but miss how different parts connect. They reviewed many studies and identified which types of risks and attacks have been tested well, like tricking the AI’s reasoning or memory in simple setups. However, more complex threats involving long-term actions, multiple agents, or human interactions are less studied. The authors suggest new questions to guide future research for stronger and more reliable AI security testing.

What this means in practice

  • For security testing teams: Design better AI security evaluations by focusing on understudied persistent and multi-agent threat scenarios uncovered in this survey.
  • For ai platform developers: Improve AI system safety features by addressing identified gaps in evaluation for long-term memory and human interactions.

A survey. It maps existing work.

Authors

Heewon Baek, Alsharif Abuadbba, Kristen Moore, Hyoungshick Kim, Surya Nepal

Abstract

Agentic AI extends LLM security beyond generated content to persistent state, autonomous actions, tool use, and interactions with humans and other agents. Existing threat classifications often emphasize individual dimensions, obscuring connections among entry points, affected components, and security consequences. The known threat landscape also differs from the coverage demonstrated by empirical research. Through a structured review of 66 studies published from 2022 to 2026, we introduce T={S, B, P, A}, a cross-dimensional representation linking affected functional or system surfaces {S}, interaction or trust boundaries {B}, violated security properties {P}, and empirically examined architectures {A}. We analyze 22 artifact-backed red-teaming studies and 11 representative security benchmarks to characterize empirical coverage and evaluation maturity. Within the selected studies, evidence concentrates on prompt/reasoning, memory, and tool-mediated attacks, predominantly in single-agent settings. Persistent, Human--Agent, complex multi-agent, systemic, and long-horizon threats receive less coverage. These findings describe the selected corpus rather than establish gaps across all empirical research. Heterogeneous metrics, limited adaptive defense evaluation, architectural imbalance, and incomplete execution-state capture further constrain comparison and reproducibility. We derive 13 open research questions to guide more systematic, architecture-aware, and reproducible security evaluation of agentic AI.