All Explanations are Wrong, But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models
2026-07-10 • Machine Learning
Machine LearningArtificial IntelligenceInformation Retrieval
AI summaryⓘ
The authors challenge the common belief that making machine-learning models explainable harms their accuracy. They propose a new idea called the Rashomon Explanation paradigm, which creates a group of trustworthy explanations that help guide the model's predictions. They also introduce RashomonLLM, a method that produces natural language explanations aligned with predictions and prove it works well. Testing on various real-world tasks, their approach improved both prediction accuracy and explanation quality compared to other methods. This work shows that explanation and prediction can work together to benefit both model performance and trust.
Explainable AImachine learningaccuracy-explainability trade-offRashomon Explanationexplanation fidelitynatural language explanationsprediction-guiding explanationsdistribution shiftsagentic workflow
Authors
Pan Li
Abstract
Explaining machine-learning models is increasingly important for decision-making and consumer trust, yet it is widely believed to come at a cost: existing Explainable AI (XAI) methods suffer from a persistent accuracy-explainability trade-off. We argue that this trade-off is not fundamental, but an artifact of treating explanation and prediction as separate objectives; when properly coupled, they become complementary, so that equipping a model to explain itself improves, rather than degrades, its accuracy. We introduce the Rashomon Explanation paradigm, which builds a set of faithful, prediction-guiding explanations rather than a single one, and prove that this set is generally non-empty and that explanation fidelity bounds the performance of the models it guides. To explore this set, we propose RashomonLLM, an Explanation-Prediction-Reflection agentic workflow that generates explanations in natural language by iteratively aligning them with predictions, and we prove it converges and recovers the full set. Across customer-churn classification, clinical survival regression, and industrial click-through prediction on large-scale live-streaming logs, RashomonLLM significantly outperforms state-of-the-art prediction and XAI baselines on both accuracy and explanation quality, with gains driven by explanation fidelity and robust to distribution shifts, temporal splits, and seeds. Our framework thus advances business performance while laying the groundwork for consumer trust.