Informational help-seeking on reddit stayed steady after chatgpt arrival
Informational Help-Seeking on Reddit Did Not Decline After ChatGPT
Social and Information NetworksComputers and Society
Summary
People did not stop asking questions on Reddit for advice even after ChatGPT, a powerful AI, was introduced. The authors compared communities where AI can answer questions with hobby groups where it cannot, during the same time. They found no big drop in people seeking help from others. Also, AI-written posts only slightly increased and couldn't explain any decline in human questions or answers. This shows humans still prefer to ask and answer questions with each other online.
What this means in practice
- •For community managers: Assess actual impact of AI tools on user engagement by comparing affected forums with control groups over time.
- •For social media platform operators: Monitor and mitigate false interpretations of AI impact on user content trends by applying comparative analysis methods.
Authors
Hazem Ibrahim, Yasir Zaki
Abstract
Did people stop asking other people for advice online once generative AI could answer their questions? Prior work on ChatGPT's effect on online help-seeking disagrees in both size and sign, in part because no study has compared affected communities against similar communities that AI cannot easily substitute for, over the same months. In this paper, we track monthly post counts in 26 Reddit informational communities against 90 size-comparable hobby communities over the same six calendar months before and after the launch of ChatGPT. We also repeat the entire analysis at 66 earlier dates, before ChatGPT existed, to see what our method reports when no ChatGPT-effect exists. We find that informational help-seeking did not decline. Our results rule out any decline in posting larger than 3.4%, far smaller than the 8% to 25% declines documented in prior work. Steady post counts could still be misleading if AI-written posts had replaced human ones. We test this possibility by scoring 274,411 posts and 223,775 comments with AI-text detectors, compared in a way that cancels out detector false-positives on human-written text. AI-written posts rose only 2-3 percentage points more in informational communities than in hobby communities, short of the 5.1 points that would be needed to hide even the smallest decline previously reported for Reddit. In addition, the comments people receive show no such rise at all. Why, then, do published studies disagree? Reddit community types were already drifting apart before ChatGPT existed, at rates comparable to every published estimate, and without same-time controls, that drift can look like an effect of generative AI. Our own largest estimate, an 18% fall in posts to low-stakes curiosity communities, matches its pre-existing trend. Humans still ask humans for help, and, as far as detection can tell, humans still answer them.