It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation
2026-07-20 • Human-Computer Interaction
Human-Computer InteractionComputation and Language
AI summaryⓘ
The authors studied different ways conversational AI can communicate to help people check facts better. They tested eight styles of AI responses that might make users think more critically, like asking questions or giving explanations. They found that explanations that guide users step-by-step led to the best improvement in accuracy, while even more challenging, debate-like styles worked somewhat well. Users liked the style that presented facts in a new way but disliked the style they felt took too long. The authors highlight the balance between helping users be accurate, keeping them happy, and encouraging thoughtful evaluation.
AI-assisted information evaluationrhetorical patternsfact verificationScaffold ExplanationSocratic Questioningalternative framingadversarial interactionsuser reflectionconversational agentscritical thinking
Authors
Sadra Sabouri, Zeinabsadat Saghi, Jordan Lee Boyd-Graber, Jonathan May, Jonathan K. Kummerfeld, Souti Chattopadhyay
Abstract
Prior work on AI-assisted information evaluation has largely focused on what AI systems communicate, comparing explanation types and formats, with responses predominantly cast in directive rhetoric where the system delivers a verdict and the user passively accepts it. While debate-style interactions have recently shown promise in prompting critical evaluation over deference, the rhetorical patterns that structure AI responses and how they might induce reflection, uncertainty, or independent reasoning remain largely unexamined. To address this, we investigated eight rhetorical patterns known to induce contemplation: Intentional Misleading, Interpretive Alternative, Scaffold Explanation, Triggering Distrust, Information Distortion, Alternative Framing, Socratic Questioning, and an Oracle baseline. Through a within-subject study with n=98 participants on a hint-on-demand fact verification task, we observed preliminary evidence that Scaffold Explanation were associated with the highest accuracy gains, and encouraging deeper reflection. Surprisingly, the adversarial conditions also improved accuracy modestly. Participants preferred Alternative Framing most and Interpretive Alternative least, largely due to the latter's perceived time cost. We discuss the implications of designing conversational agents with varied rhetorical styles and the trade-offs among user performance, satisfaction, and contemplation.