Chatbot corrections reduce false sharing but do not boost long term learning
Chatbot Engagement Does Not Always Beget Metalearning: Evidence from Three Countries
Human-Computer Interaction
Summary
People often believe false information from images shared online. The authors studied if chatting with a fact-checking chatbot helps people not only correct false beliefs right away but also learn to avoid them later without help. They tested this in the US, India, and Singapore with about 2,200 participants. While chatbot conversations immediately helped people recognize false claims, the effect did not last a week, and sometimes the chatbot group did worse later. This means quick chatbot help alone does not create lasting understanding.
What this means in practice
- •For social media platform teams: Design real-time chatbot fact-checks that reduce immediate sharing of false image claims across diverse countries.
- •For public communication strategists: Incorporate chatbot corrections to lower false information spread but rely on additional strategies for lasting awareness.
Authors
Kokil Jaidka, Insyirah Binte Imam Mujtahid, Peng Qi, Harshit Aneja, Subhayan Mukerjee, Wynne Hsu, Mong Li Lee, Tsuhan Chen
Abstract
Chatbots deliver real-time fact-checks, but whether a chatbot correction leaves anything behind once the chatbot is gone - metalearning, distinct from correcting misbeliefs - is untested. We report a preregistered, three-country randomized experiment (USA, India, Singapore; N ~ 2,200) on out-of-context image misinformation, manipulating a correction's channel affordances (synchronicity, bandwidth) across four conditions: Control, Links-only, Static explanation, and a Socratic Chatbot built on a validated out-of-context detector, with an unaided retest one week later. The Chatbot produced the largest immediate discernment gain (d = 0.097, p = .023). All three interventions reduced sharing of false claims (d ~ -0.12, p < .01). One week later, no advantage persisted: the Chatbot arm declined relative to Control, most sharply in India and Singapore, and in India on claims it never discussed. Decay tracked affordance level and did not vary by country. Engagement mechanisms, we argue, do not substitute for slow AI literacy.