Comment-level Topic Drift Analysis in the Reddit Corpus
2026-08-19 • Computation and Language
Computation and Language
AI summaryⓘ
The authors used advanced language models to study how topics in Reddit comments changed over time from 2006 to 2022. By turning comments into meaningful number patterns, they tracked how groups of related topics evolved and drifted in meaning. They created a new way to measure these changes in topic discussions at a very detailed level and checked their results against a baseline to ensure accuracy. Their findings show that political and social topics tend to shift a lot, while topics like music and sports stay more stable.
embedding-based topic modelingsemantic embeddingstopic driftpretrained language modelsReddit commentsunsupervised learningdiscourse evolutionnull model
Authors
Steven Morse, Daniel Runfola, Trenton W. Ford
Abstract
We present a novel application of embedding-based dynamic topic modeling techniques to detect and quantify topic drift at the comment level in a massive corpus. By leveraging pretrained language models to generate contextualized semantic embeddings for short text, we analyzed 12.7 billion Reddit comments spanning 2006 to 2022. Using unsupervised methods on these embeddings, we identify dynamically evolving topic clusters over time. Our primary contribution is a methodology for analysis of semantic drift and discourse evolution in the embedding space itself. We also demonstrate modifications to existing methods that enable this analysis at scale, and we propose and demonstrate a null model comparison test to filter spurious dynamics. Key findings suggest that politically and socially contentious topics exhibit significant directional drift in embedding space, with inter-topic distances changing systematically over time beyond what the null model can explain, whereas domains such as music and sports remain comparatively stable.