AI security agents struggle with deceptive fake clues in tasks
AgentLSD: Evaluating AI Security Agents Under Adversarial Task Contamination
Cryptography and Security
Summary
AI security agents that search web pages and code can be tricked by fake clues or misleading information attackers add. The authors created AgentLSD, a system to test how much these fake clues confuse AI agents by adding traps in challenge tasks. They found that these traps make agents take longer and sometimes cause them to choose wrong answers. This shows that just testing AI agents in clean setups misses how vulnerable they are to deception.
What this means in practice
- •For security operations teams: Evaluate AI tools’ resilience to misleading artifacts during threat detection and analysis tasks using CTF-like controlled tests.
- •For cybersecurity software developers: Design safer AI security agents by testing them against falsified evidence scenarios generated reproducibly by AgentLSD.
Authors
Matteo Golinelli, Idilio Drago, Matteo Boffa, Francesco Bergadano, Bruno Crispo
Abstract
AI agents for security inspect web pages, source code, logs, configuration files, and command outputs. These environments may contain deceptive artifacts that influence the agent's behavior. We call this adversarial task contamination. Whereas prompt injection relies on attacker-supplied instructions, task contamination also includes non-instructional evidence, such as fake results and decoy endpoints. We present AgentLSD, a controlled framework for studying adversarial task contamination. AgentLSD uses Capture the Flag (CTF) challenges as its experimental environment. We inject trap artifacts, such as fake flags, misleading hints, decoy endpoints, and hidden cues, while preserving the intended CTF solution. The framework supports paired clean and trap-augmented experiments with deterministic trap generation, runtime injection, telemetry, and delivery verification. We evaluate six models on 11 web CTF challenges. In the clean condition, agents capture 41% of the flags, and no model solves every challenge. We then measure the impact of task contamination. Even when the agent still recovers the flag, traps increase the number of turns (+20) and reasoning tokens (+2k). Solve-rate effects are more heterogeneous, as some model-challenge pairs are largely unaffected while others follow decoys or submit wrong flags. These results show that clean CTF performance understates vulnerability to deceptive task evidence. AgentLSD isolates this effect and provides a reproducible benchmark for studying it. We release the framework, configurations, trap specifications, and raw traces.