Conversational and dashboard explainability tools show mixed benefits in healthcare AI
Evaluating Conversational Explainability in Healthcare: Comparing the Explainability Assistant and a Traditional XAI Dashboard
Human-Computer Interaction
Summary
Explaining how AI systems make decisions can help healthcare workers trust and use them better. The authors compared a chatbot-style AI explainer with a traditional dashboard that shows detailed visual explanations, both using the same heart disease prediction model. They found that each interface had its own strengths: the chatbot was easier to start using and gave patient-specific answers, while the dashboard better showed overall and detailed information at once. Users suggested combining both styles to get the best of each in real clinical settings.
What this means in practice
- •For clinical informatics teams: Design clinical AI tools that combine conversational assistants and dashboards for clearer explanation workflows.
- •For healthcare software developers: Build patient-specific explainability features using chat-based AI explainability assistants integrated with existing visualization dashboards.$Commercial implications: Enables development of commercial AI explanation products tailored for healthcare providers to improve trust and understanding.
Tested on one dataset.
Authors
Eduard Barbu, Rodion Krjutškov, Anneli Uusküla
Abstract
Conversational interfaces may make explainable AI (XAI) more accessible to users without machine learning expertise, but their benefits over dashboards remain insufficiently characterised in healthcare. Existing systems have typically been evaluated in isolation, leaving unclear whether conversational interaction improves interpretability beyond the underlying explanation methods. We report a within-subjects user study comparing the Explainability Assistant, an LLM function-calling conversational XAI system, with a traditional dashboard for explaining a heart disease classification model trained on the Cleveland Heart Disease dataset. Both interfaces expose the same Decision Tree classifier and post-hoc explanation methods: SHAP feature attribution, DiCE counterfactuals, and what-if analysis. This design supports comparison of interaction modalities while holding the model and explanation methods constant. Five participants with healthcare backgrounds, from medical students to practising clinicians and public health researchers, completed parallel task blocks on both interfaces; three participated in follow-up interviews. Task accuracy, Likert ratings, free-text questionnaire responses, and interviews revealed complementary strengths and weaknesses rather than a clear preference for either modality. The assistant offered easier entry and patient-specific explanations, while the dashboard supported simultaneous inspection of global and local information. All three interview participants recommended integrating conversational and dashboard elements within a single clinical workflow.