LLM agents tested on many Linux privilege escalation cases reveal varied risks
PrivEscalate: Measuring and Augmenting the Threat of LLM-Automated Linux Privilege Escalation
Cryptography and Security
Summary
Linux privilege escalation is a crucial step hackers use to gain full control over a computer once they have some access. The authors found that previous tests of AI language models doing this were too small to understand their true abilities. They created a large set of 531 simulated Linux hacking challenges to see how well different AI models perform. Their results show models do well in some areas but not others, and small changes in the environment can make attacks fail. They also built an improved AI helper that plans better and works more successfully without changing the AI itself.
Linux privilege escalationLarge language modelsCyber kill chainPost-exploitationDockerVulnerability classesExploit sensitivityAgent architecturesReAct agentRed team training
Authors
Yixuan Liu, Zilong Zhen, Yin Wu, Yi Li
Abstract
As Large Language Model (LLM) agents increasingly automate offensive operations across the cyber kill chain, their efficacy in complex local post-exploitation tasks remains inadequately quantified. Among these, Linux privilege escalation is a key step between initial access and full system compromise. However, existing evaluations for this task are limited by small sample sizes (fewer than 15 scenarios), lacking the scale to compare model capabilities under executable verification. To address this, we present PrivEscalate, a large-scale benchmark for Linux privilege escalation, comprising 531 Dockerized scenarios spanning 14 sub-categories. We additionally derive 329 parameterized variants to measure sensitivity to environmental distractors. Evaluating six LLMs across three agent architectures reveals: (i) model capability is heterogeneous across vulnerability classes, with no single model dominating across the high-prevalence classes, motivating multi-dimensional risk assessments; (ii) LLM successes are sensitive to environmental perturbation, so configuration rotation can disrupt some exploit attempts but does not eliminate the measured risk; and (iii) agent architectures can materially change success rates and reorder model rankings, though the magnitude is model-dependent. Leveraging these insights, we develop PrivEscAgent, a domain-specialized wrapper that augments a generic ReAct agent with deterministic enumeration, category matching, and step planning. PrivEscAgent improves over prior Linux privilege-escalation agent baselines without underlying LLM modifications. We release PrivEscalate as an open-source, Dockerized measurement instrument supporting LLM agent evaluation, defensive tool validation, and red-team training.