Llm assisted digital forensics vulnerable to attacker planted text injections
Prompt Injection Through Forensic Evidence: A Taxonomy and Proof of Concept of Anti-Forensic Attacks on LLM-Assisted Digital Investigations
Cryptography and Security
Summary
Digital forensics experts now use large language models (LLMs) to help analyze digital evidence. The authors found that attackers can hide tricky text inside evidence that fools both human examiners and these AI models, causing misleading conclusions or even malicious actions on the analyst's machine. They created a system to classify and demonstrate these attacks and suggested ways to reduce the risks. This work shows new challenges when AI tools analyze potentially harmful digital evidence.
What this means in practice
- •For digital forensics teams: Detect and defend against malicious text embedded in digital evidence that can mislead AI-assisted analysis tools.
- •For cybersecurity operation centers: Improve incident response workflows by identifying when forensic AI tools may be compromised through crafted textual evidence.
Authors
Frank Breitinger, Jan Gruber, Christopher Hargreaves, Jan-Niclas Hilgert, Gaëtan Michelet, Mark Scanlon
Abstract
Large language models (LLMs) are entering digital forensics as examination assistants and autonomous agents. Digital forensics inverts the usual setting for indirect prompt injection: the evidence under analysis is attacker-controlled by definition, and the examiner is obliged to read it closely. This makes the reasoning layer a new anti-forensic target, next to the classical ones of data and tools. Classical anti-forensic taxonomies do not cover it, and prompt injection defences do not anticipate it. We extend the anti-forensic model to reflect the manipulation of reasoning, and present a taxonomy of attacks carried through adversarial evidence: an attacker selects an objective, a technique, a payload location, and a concealment method. We observe a perception gap, i.e., the fact that deterministic tools, the human examiner, and the model perceive the same evidence differently, which is what lets an injection reach the model unseen by the examiner. We demonstrate three attacks against an LLM-assisted analysis pipeline, changing a reported conclusion, writing attacker-chosen content to the analyst's workstation, and causing outbound network traffic, each driven only by text planted on the seized device. Mitigations to these attacks are also proposed.