Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection
2026-07-27 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors studied how large language models (LLMs) used for analyzing security logs can be tricked by attackers who insert misleading information into the logs. They created a method to test these trick attacks and found that LLMs often mistakenly label harmful activity as safe. However, the authors also discovered that the explanations given by the models often reveal signs of these attacks, which could help in detecting them. This work highlights potential risks and ways to spot attacks when using LLMs in cybersecurity.
Large Language ModelsSecurity Operations CenterLog AnalysisAdversarial AttacksPrompt InjectionCybersecurityIndicator of CompromiseAttack Detection
Authors
Max Landauer, Florian Skopik, Markus Wurzenberger, Franciszek Górski, Mateusz Krzysztoń
Abstract
Large Language Models (LLMs) are increasingly integrated into Security Operations Center (SOC) workflows, where they support analysts in tasks such as the interpretation of system logs. However, the ability of LLMs to directly process untrusted textual input also introduces new attack surfaces. In particular, attackers can inject contextual information or explicit instructions into log entries in order to influence how malicious activity is interpreted by the model. Despite the growing adoption of LLMs for log analytics, the robustness of such systems against adversarial log injection remains largely unexplored. To address this gap, this paper presents a framework for evaluating prompt injection attacks against LLM-based log interpretation. Using log traces generated during real cyber attacks, our approach creates adversarial examples through generic injection generation, refinement, and attack-specific optimization. Our evaluation across multiple state-of-the-art LLMs shows that these injections can cause malicious log traces to be classified as benign despite containing clear indicators of compromise. As a potential remedy, we show that the explanations generated by the LLMs alongside their classifications frequently contain indicators of adversarial manipulation that can be leveraged to detect such attacks.