Large language model unlearning reduces knowledge recovery risks
Cascade: Hierarchical Recoverability Control for Large Language Model Unlearning
Computation and Language
Summary
It can be important to remove certain knowledge from large AI language models, like private or copyrighted information, without breaking the model's ability to work well. The authors present a new approach called Cascade that stops this sensitive knowledge from being recovered later. Cascade uses a combination of methods to block, compress, and intervene in how the model processes and outputs information related to that knowledge. Tests show this method reduces the chance of retrieving the removed information while keeping the model useful for general tasks.
What this means in practice
- •For machine learning engineers: Control which knowledge is retained or removed in language models to protect data privacy while maintaining performance.
- •For ai ethics teams: Implement safeguards to remove sensitive or copyrighted information from AI models to comply with privacy regulations.
Authors
Qingchen Yu, Shiying Duan, Xiaodong Li, Yuhua Wang, Zhiyu Li, Shiji Zhou, Yifan Sun, Zhaoxin Fan
Abstract
Large Language Model (LLM) unlearning is essential for removing sensitive or copyrighted knowledge while preserving general utility. Existing methods often leave residual knowledge in intermediate representations, which can still be recovered. To address this, we propose Cascade, a hierarchical recoverability control framework that minimizes the internal identifiability of target knowledge. Cascade combines three complementary controls: path-level routing to suppress privacy-associated activation routes, representation-level compression to reduce geometric separability, and decoding-level intervention to limit residual recovery. Experiments on TOFU, MUSE-News, and WMDP, including robustness tests with query reformulation and extraction-style prompts, show that Cascade effectively reduces recoverability while maintaining stable model utility.