Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs
2026-06-08 • Cryptography and Security
Cryptography and SecurityInformation TheoryMachine Learning
AI summaryⓘ
The authors studied how big language models can hide secret messages in their outputs, which is hard to detect by just looking at the output text. They tested a known method that tries to find these secrets by examining the model's inner workings, but showed that sneaky attackers can trick this method using special fine-tuning. To counter this, the authors introduced a new approach involving targeted adjustments to the data the model sees, which helped reveal hidden secrets again. Their work shows both that detecting hidden messages inside models is tricky and that smart testing methods can help find them.
large language modelssteganographyfine-tuninglinear probesMLP probessecret recoveryadaptive evasioninformation theoryactivation-based detectionmechanistic interpretability
Authors
Charles Westphal, Timothy Douglas, Keivan Navaie, Tiago Pimentel, Fernando E. Rosas
Abstract
Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs. This creates a steganographic exfiltration risk that is difficult to detect with output-level steganalysis. Recent work proposes mechanistic detection using linear probes that recover the secret from internal activations. We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention. First, we extend the detection setup to include a non-linear MLP probe. We then adversarially fine-tune steganographic trojans across five base models: Qwen3-8B, Llama-3.1-8B, Ministral-8B, Qwen3-14B, and Phi-4-14B. The resulting models retain $58$--$79\%$ exact-match secret recovery while evading both ridge and held-out MLP probes, with $1$--$8\%$ average capability degradation across six benchmarks. We then give an information-theoretic characterization of this evasion. Successful evasion preserves recoverability while reducing low-order extractability of the secret from the content-aligned representation, forcing the payload into synergistic interaction with residual degrees of freedom. This motivates a recontextualization dataset that restricts these residual degrees of freedom. On this distribution, both ridge and MLP detectability are restored across all five evasive trojans. Overall, our findings show that activation-based steganography detection is vulnerable to adaptive evasion, but also that theory-guided evaluation distributions can expose otherwise hidden payloads.