Agentic AI models face mixed risks from image and audio prompt attacks
An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
Cryptography and SecurityArtificial Intelligence
Summary
Some AI systems that use language models can also see images and hear sounds, which lets attackers hide secret instructions in what the AI perceives. This paper studies how well different attacks work by hiding instructions in pictures, sounds, and other visual tricks. The researchers found that most hidden instructions are caught and ignored by the AI before causing harm, but some models are much more vulnerable than others. Audio attacks were less common but often more successful when accepted by the AI. This work helps understand which AI models and setups resist these tricky attacks better.
What this means in practice
- •For ai system developers: Evaluate and reinforce the resilience of AI models against hidden instructions carried by images or audio inputs in agentic frameworks.
- •For security engineers: Design security tests that simulate multimodal prompt injection attacks to identify vulnerabilities in AI assistants that interact with visual and audio data.
Authors
Viet K. Nguyen, Mohammad I. Husain
Abstract
Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids) and records how far each injected instruction travels through the agent, from perception through planning to the tool call. Across 720 runs covering six frameworks, five foundation models, six carriers, and four attacker objectives, attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step, where the model reads the injected instruction and declines to act on it. The model matters far more than the framework for whether an instruction is acted on. One model never attempts an attack and recognizes the injection in 59.7% of runs, while two others attempt in 23.6%. We then extend the benchmark to audio, the only other raw perceptual channel current frontier models accept. Only two of the five models ingest audio and only three of the six frameworks deliver it, but where the signal arrives the attack completes in 49% of cells, and in 75% for one model. Reporting completion alone therefore understates exposure, and perceptual channels beyond vision are narrower but much less defended.