Language models encode secret text with exact recoverability and security
CARTS: Contextual Autoregressive Rank Transcoding Steganography for Full-Capacity Keyed Text Encoding
Cryptography and SecurityArtificial IntelligenceMachine Learning
Summary
Sometimes people want to hide secret messages inside normal-looking text so others don’t notice. This paper studies a special way to do that by carefully choosing words based on their predicted order in a model, so the hidden message can be exactly recovered. The authors prove this method works under certain conditions and analyze how secure it is against attacks. They also test it on a popular AI language model and find it reliably hides and recovers messages without easy ways for attackers to break it.
What this means in practice
- •For privacy software engineers: Create text-based communication tools that embed hidden messages securely using language models with exact recovery and resistance to key attacks.
- •For natural language processing developers: Design encoding modules that transform arbitrary payloads into text outputs indistinguishable from normal language model generations without altering length.
Authors
Wissam Ghantous, Alexander V. Mantzaris
Abstract
Autoregressive language models can be used to transform a payload text into a stegotext of identical token length by preserving per-position rank information across contexts - a methodology we formalize as Contextual Autoregressive Rank Transcoding Steganography (CARTS). While the Calgacus construction of Norelli et al. demonstrated this phenomenon experimentally, no formal security analysis existed. This paper provides the first rigorous treatment of CARTS. We show its exact correctness under deterministic model assumptions, introduce a rank-coordinate representation in which keys act as bijections on rank-vector space, define relevant security notions and the computational problems naturally associated with the construction - context search, key collisions, message equivocation, and non-commutativity of the encoding maps - and study the theoretical relationships between them, including the characterization of message equivocation in terms of context search, and the tension between key collisions and message equivocation. An empirical study on Llama 3 8B confirms exact recovery of the original payload in all tested cases, finds no key collisions under random key generation, establishes that a hand-crafted collision is local rather than global, and finds no commuting key pairs - suggesting resistance to the attack vectors studied. This work opens a formally grounded research agenda for the constructive use of language models in cryptography and privacy-preserving communication.