Large language models can rate explainable AI explanation quality
XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?
Artificial Intelligence
Summary
It is hard to measure how good explanations from explainable AI methods are because people judge them differently. The authors studied if large language models (LLMs) can consistently and fairly compare these explanations. They created XAI-Arena, a system where LLMs act like judges to evaluate explanations on many aspects such as clarity and trustworthiness. Their tests showed that the LLM ratings strongly matched human opinions, making LLMs a useful tool for scalable and repeatable assessments.
What this means in practice
- •For machine learning engineers: Use LLM-based evaluations to compare explanation methods across models and datasets efficiently and reproducibly.
- •For software quality assurance teams: Assess and benchmark AI explainability features for products targeting different user needs without relying solely on user studies.
Authors
Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
Abstract
Evaluating the quality of explanations produced by explainable AI (XAI) methods remains challenging because existing approaches often rely on subjective human judgment, limiting reproducibility, scalability, and comparability between studies. We examine whether LLMs can serve as a reproducible and scalable mechanism to make comparative assessments of the quality of XAI explanations. We introduce XAI-Arena, an LLM-as-a-judge framework for scalable, reproducible, multidimensional, and stakeholder-sensitive evaluation of XAI explanation quality. XAI-Arena then allows us to compare XAI explanations along various dimensions, namely, perceived simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability. We then benchmark XAI explanation methods across various datasets, machine learning models, and stakeholder personas. Human validation shows a strong positive association between LLM-generated and human ratings (Spearman's rho=.693, p<.001). Together, LLM-based evaluations can capture systematic differences in XAI explanation quality and provide a scalable and reproducible framework for comparative assessment of XAI explanations.