Limitations of logprob metrics for personal data suppression in large language models
Reducing Logprob Without Causal Verification: Diagnosing SAE-Based PII Suppression in LLMs
Artificial Intelligence
Summary
Large language models can accidentally reveal private information, so researchers try to stop that by tweaking parts of the model. This paper finds that the usual way of checking if the tweaks work, which looks at how likely the model thinks certain words are, can be fooled by unrelated effects. The authors show that many tweaks thought to work don’t actually target the private information paths in the model. They suggest a new way to stop leaks by carefully finding the exact parts of the model responsible and testing cause and effect more deeply.
What this means in practice
- •For language model engineers: Improve methods to evaluate and design interventions that reliably suppress private information leaks in language models.
- •For data privacy teams: Develop safer NLP systems by guiding privacy efforts toward causally based suppression rather than heuristic logprob checks.
Authors
Chaerin Lee, Wonho Lee, Sohee Park, Daeseon Choi
Abstract
SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations. Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics. While 17 cases (47%) appear successful under logprob-only evaluation, only 1 case (2.8%) passes our multimetric framework. The rest reveal two failure modes: random directions outperform selected features, and feature selection collapse across PII types. These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage. A 9,000 run causal mediation feasibility study corroborates this: the verifier separates candidates reliably, but correlation-selected candidates rarely pass it. We therefore propose a new research mission for NLP: causally grounded PII suppression through circuit-level localization and causal mediation analysis.