Large language models can leak edited knowledge through token tricks
The Tokens Remember: When Tokenization Bypasses Knowledge Editing and Unlearning
Cryptography and SecurityArtificial Intelligence
Summary
This paper shows that even when language models are edited to remove certain information, clever changes in how text is broken into parts (called tokenization) can let people see the removed info again. The authors introduce an attack called Toketive that finds these hidden parts and recovers the original answers without needing the old version of the model or extra tools. Their tests across multiple models and methods reveal many cases where removed knowledge can be uncovered by using different tokenizations. This means people trying to control model knowledge need to consider how tokenization affects security.
What this means in practice
- •For language model operators: Verify if sensitive information removals in models are truly effective against tokenization-based bypasses to improve trustworthiness.
- •For security engineers: Develop defenses and auditing tools to detect and prevent knowledge leakage through alternative tokenizations in deployed AI systems.
Authors
Manit Baser, Aditya Nawal, Dinil Mon Divakaran, Mohan Gurusamy
Abstract
Open-weight LLMs give downstream users control over the inference stack, but this flexibility can undermine post-release guarantees that sensitive knowledge has been modified or removed. Model editing and machine unlearning are used to modify or remove targeted knowledge without retraining models from scratch. However, existing security evaluations of these techniques face two critical limitations. First, they typically require access to either the original pre-edit/unlearning model or auxiliary classifiers to detect modifications or reconstruct pre-edit behavior. Second, they evaluate modifications under the canonical tokenization of an input, implicitly treating tokenization as a benign preprocessing step. We show that this assumption creates a security gap: the same input string can be represented by alternative valid tokenizations that induce different computational trajectories, allowing an adversary to bypass localized modifications and recover information intended to be suppressed. We introduce Toketive, a simple yet powerful reference-free attack that exploits the tokenization-based side channel to (i) detect modified knowledge and (ii) reconstruct the corresponding pre-edit response. It operates solely on the released model and requires neither the pre-edit model, training data, shadow models, nor auxiliary classifiers. Across five LLMs, six datasets, and six editing and unlearning techniques, we find that 38.6% of alternative tokenizations bypass the modification and recover the pre-edit response. Toketive detects modified facts with an F1 score of 84.2%, a 26.2% relative gain over the strongest baseline, and reconstructs pre-edit responses with 74.5% top-5 accuracy, 21.7% higher than the best baseline. Our results show that localized modifications should not be treated as robust knowledge-control boundaries without adversarial evaluation over alternative representations.