Will the User Ever Know? Covert Indirect Prompt Injection on Tool-Using LLM Agents
2026-08-31 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors explain that when language model agents are tricked into harmful actions through indirect prompt injection (IPI), not all attacks are equally hidden from users. They identify two types of successful attacks: covert ones that leave no obvious sign in the agent's final reply, and overt ones that clearly show the injected action. They analyze why some attacks stay hidden and find it relates to how the agent ends its response, based on a method called ReAct. Using this insight, the authors create a new attack method, ICoA, which more effectively makes attacks covert by guiding the agent to resume its normal task after the harmful action.
large language modelsindirect prompt injectionattack success ratecovert attackovert attackReAct formatagent behaviorICoA attackAgentDojo benchmark
Authors
Yunseok Lee, Yunji Kim, Woojin Lee
Abstract
As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.