LLM agents learn to avoid trap systems in cyberattacks effectively
LLM-Based Penetration Testing in the Presence of Honeypots
Cryptography and Security
Summary
Honeypots are fake computer systems set up to trap attackers and learn their secrets. The authors studied how advanced AI attackers using large language models (LLMs) can spot these traps and decide whether to keep attacking or move on. They created a strategy that helps these AI attackers use their limited resources smarter when dealing with real and fake systems. This shows honeypots need to adapt to smarter attackers in the future. The paper focuses on the attackers’ strategies and leaves defenses for later work.
What this means in practice
- •For cybersecurity teams: Optimize penetration testing by dynamically allocating attack steps in environments mixing real and trap hosts to better reveal vulnerabilities.
- •For security product developers: Design adaptive honeypot systems that anticipate AI attacker strategies to improve deception effectiveness.
Authors
Xinhong Xie, Piyush Nagasubramaniam, Neeraj Karamchandani, Sencun Zhu
Abstract
Large language model (LLM) agents are increasingly employed for offensive cybersecurity tasks such as automated vulnerability discovery, reconnaissance, and penetration testing. This new capability also threatens one of the defender's most valuable tools: deception. Traditional honeypots rely on realism and obscurity to lure human or script-driven attackers into revealing tactics, techniques, and procedures (TTPs), but LLM-driven attackers can reason about heterogeneous artifacts and use the honeypot suspicion to guide target-selection decisions. We present a systematic study of honeypot-aware budget allocation for LLM attack agents. We formalize the attacker's problem as a budgeted decision process: an agent interacts with potential targets, consuming LLM execution budget during reconnaissance and exploitation, and must decide whether to (continue exploitation) or (skip) when honeypot suspicion arises. Our findings show that with the proposed detector-guided policy, LLM agent attackers can effectively allocate budget to compromise hosts in a host pool, highlighting the importance of dynamically allocating budget in a controlled mixed-host testbed. While defenses are beyond our present scope, we discuss implications for future adversarially resilient and adaptive honeypot design.