Model size shapes quality of counterfactual explanations in language AI
An Empirical Study of Counterfactual Self-Explanations in LLMs
Computation and Language
Summary
Large AI language models can explain why they give certain answers by changing parts of the input to flip their predictions. The authors studied this with ten different models on tasks like sentiment and logic. Bigger models made better explanations that matched human reasons more closely. However, trying to guide explanations with human-like hints improved clarity but didn't always improve how truthful the explanations were. So, explanations from these models can help understand their decisions but aren’t always reliable unless checked carefully.
What this means in practice
- •For ai developers: Evaluate and improve explanation methods for large language models to better understand model decisions in NLP tasks.
- •For product managers in ai: Assess the reliability of AI self-explanations to build more transparent and trustworthy language-enabled applications.
Authors
Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Vassilis Lyberatos, Giorgos Stamou
Abstract
Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes. Across sentiment analysis and natural language inference, we evaluate ten instruction-tuned models from the LLaMA-3 and Qwen-2.5 families, measuring faithfulness, minimality, and alignment with human-annotated rationales. Our results show that model scale is the strongest determinant of explanation quality: larger models are substantially more likely to generate counterfactuals that flip their own predictions and target decision-relevant evidence. In contrast, the rationale-guided condition produces edit-minimal counterfactuals that are also more human-aligned. However, it does not consistently improve faithfulness. Overall, counterfactual self-explanations can provide useful behavioral evidence about model decisions, but their reliability depends strongly on model capacity and should be empirically validated rather than assumed.