Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision
2026-06-30 • Computation and Language
Computation and LanguageArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors studied when language models (LMs) trained to explain their own predictions genuinely reflect their behavior rather than just copying given explanations. They found that even when trained on fixed explanation data from earlier versions or different models, the LMs often generate explanations that match their current behavior better than the training examples. This happens because explanations remain connected to the model’s behavior over time, adapting as the model changes. The effect occurs across different tasks and is robust even with noisy labels, suggesting fixed explanation datasets can help models explain themselves more faithfully after training.
language modelsexplanationscounterfactual behaviorintrospectionpost-trainingmodel behaviorsycophancyrefusallabel noisetraining supervision
Authors
Zifan Carl Guo, Laura Ruis, Jacob Andreas, Belinda Z. Li
Abstract
When does training language models (LMs) to generate explanations of their predictions yield faithful introspection, rather than superficial imitation? We study LMs trained to explain which features of their inputs influenced their behavior, using models' counterfactual behavior on modified inputs as supervision. Surprisingly, we find that LMs trained on fixed counterfactual explanations derived from earlier checkpoints of themselves, or even from behaviorally similar models in different families, frequently produce explanations more faithful to their own current behaviors than to those of their training targets. This "introspective" coupling between LM explanations and behaviors occurs when training explanations remain sufficiently correlated with current behaviors over the course of training, even as behaviors themselves shift. We also show that introspective coupling tracks behavior shifts: when explanation training is provided concurrently with other post-training objectives, explanations track those shifts without requiring updated supervision. This phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise. Overall, our results show that even fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.