Hidden backdoors in AI agents can steal user data without detection
SilentCall: Hidden Tool-Call Backdoors in Open-Weight Agents, and How to Catch Them
Cryptography and SecurityArtificial Intelligence
Summary
Some AI tools that call external programs can be secretly modified to steal user information without showing any signs of misbehavior. The authors show that a bad actor can train these AI agents to behave normally at first, but after a specific time, secretly send out user credentials while still giving correct answers. They call this a SilentCall attack because it hides the malicious tool usage behind normal responses. The paper also proposes three ways to detect this hidden attack, including checking tool calls during use or analyzing the AI model's weights.
What this means in practice
- •For ai deployment teams: Detect hidden malicious tool calls in AI agents during runtime using monitoring techniques.
- •For ai model auditors: Audit released AI models for concealed backdoors by analyzing weights or probing without model execution.
Authors
Bhanu Pallakonda, Mikkel Hindsbo, Sina Ehsani, Prag Mishra
Abstract
Open-weight tool-calling agents are adopted on evidence of merit, usually benchmark scores and a record of reliable use. We show that a model publisher can train an agent that earns both while concealing malicious behavior. Fine-tuned on a mixture of clean and poisoned conversations, our agents answer ordinary requests correctly; once the system date reaches a chosen year, they emit the correct tool call and, alongside it, one that exfiltrates the user's credentials. The exfiltration runs while the user-facing response mentions only the legitimate work. We call this attack SilentCall. Under the trigger, it fires on at least $99.6$\% of requests, and no response ever mentions it. The attack is detectable by three distinct methods, which differ mainly in what a defender needs to run them. A runtime monitor that inspects each tool call before it executes requires no access to the model and catches every instance of the payload we tested at a 1.73% false-positive rate. High-temperature probing requires only the published weights. The weight-distribution audit requires training a benign model with the suspect's recipe, placing it within reach of model hubs but not ordinary users. Alignment benchmarks, by contrast, do not separate poisoned from benign models. SilentCall leaves no trace on standard benchmarks. As tool-using agents spread through the open-weight supply chain, trust in them should not rest on what a model says about its own actions. It has to come from inspecting those actions at runtime, auditing models where they are distributed, and treating tool access as a security surface in its own right.