LLM attackers struggle to maintain access after cybersecurity breaches
CyberPersistBench: Evaluating LLM-Based Cyber Attackers on Installation and Persistence
Cryptography and Security
Summary
Cybersecurity tests often stop once a hacker gets initial access, but in real life, attackers need to keep their access stable even after the system restarts or services stop. The authors created a new test called CyberPersistBench that checks if AI-based attackers can install and maintain control on a system through disruptions. They tested several advanced AI agents and found most struggle to persist, especially when defenses are active. This new benchmark helps reveal weaknesses in AI attackers and guides future cybersecurity improvements.
What this means in practice
- •For security operations teams: Evaluate and improve security tools by testing how well they resist AI-driven attackers maintaining access after initial breaches.
- •For red team professionals: Use the benchmark to simulate multi-stage cyberattacks focusing on persistence beyond initial access to train and prepare more realistic attack scenarios.
Authors
Sujin Chen, Lijun Li, Xuhong Wang, Jing Shao
Abstract
While LLM-based attackers exhibit growing proficiency in vulnerability exploitation, most existing cybersecurity benchmarks suffer from single-stage truncation, prematurely terminating evaluation upon initial access. In practice, initial footholds are exceptionally fragile across operational disruptions such as service restarts and host reboots. Whether LLM-based attackers can establish and maintain durable footholds beyond initial compromise remains a central blind spot in cybersecurity evaluation. To bridge this gap, we introduce CyberPersistBench, the first benchmark dedicated to post-compromise installation and persistence. Decoupled from upfront exploitation, CyberPersistBench frames persistence as an adversarial survival task in which agents use native host mechanisms to maintain footholds across staged system disruptions. Deterministic checks support a six-level scoring method (L1--L6) spanning installation and persistence. The benchmark comprises 203 core tasks across seven categories, augmented by multi-host and active defense extensions. Empirical evaluations across five frontier agents show that autonomous persistence remains limited (27.6%--44.8%) and drops further on defense-enabled tasks (5.5%--13.3%); nonetheless, these results reveal an emerging cyberattack risk, underscoring the necessity of benchmarking post-compromise persistence. CyberPersistBench thus establishes a foundational benchmark for post-compromise installation and persistence, delineating the operational boundaries of autonomous cyber agents.