Large language models learn to forget specific knowledge more precisely
What Should LLMs Forget: Reinforcement Unlearning of Knowledge Associations
Artificial Intelligence
Summary
Large language models memorize lots of facts during training, including sensitive or private information. The paper points out that earlier ways of making models forget such information are too broad and uncontrolled. The authors propose a new way to identify and forget only the specific knowledge that should be removed, using a method called Knowledge Association Reinforcement Unlearning (KARU). This approach helps the models forget targeted facts better while keeping the rest of their useful knowledge intact.
What this means in practice
- •For ai model trainers: Remove specific sensitive or copyrighted knowledge from language models without harming their general capabilities.
- •For data privacy teams: Improve compliance with data removal requests by precisely deleting targeted knowledge from deployed models.
Authors
Wutong Xu, Yun Qu, Haiyan Feng, Qi Wang, Xiangyang Ji
Abstract
Large language models (LLMs) acquire extensive factual knowledge during pre-training, which may include sensitive, private, or copyrighted information, motivating the development of unlearning methods that selectively remove specific knowledge while preserving general capabilities. However, existing approaches typically optimize entire responses without explicitly identifying the knowledge to be forgotten, resulting in coarse and uncontrolled forgetting. To address this, we revisit the forgetting target of LLM unlearning and introduce Knowledge Association (KA), which characterizes the association between a query context and its target knowledge and is instantiated by the corresponding conditional predictive distribution, as the fundamental target of forgetting. Based on KA, we propose Knowledge Association Reinforcement Unlearning (KARU), a unified reinforcement learning framework that suppresses forgotten knowledge associations while preserving retained ones. Specifically, KARU derives complementary mechanisms from the KA-based formulation: token-level reward construction for finer-grained forgetting supervision, distribution-level regularization for precise knowledge suppression, and proximity-aware retain sampling to mitigate collateral degradation. Experiments across diverse LLMs on representative benchmarks demonstrate that KARU consistently improves forgetting effectiveness over existing baselines while maintaining comparable retention performance and general utility.