Illinois corpus tracks social attitudes on Reddit from 2007 to 2023
The Illinois Social Attitudes Aggregate Corpus (ISAAC): An Open Tool and Reproducible Pipeline for Analyzing Social Group Discourse at Scale
Computation and LanguageSocial and Information Networks
Summary
Understanding how people talk about social groups online is hard because the data is scattered and noisy. The authors created ISAAC, a huge collection of over 527 million Reddit posts filtered to focus on six social group topics such as race and age. They added labels for emotion, sentiment, and morality, and linked posts to users’ locations, making it easier to study changes over time and across places. This tool helps researchers compare discourse and track how social attitudes shift during major events.
What this means in practice
- •For social media analysts: Analyze trends and shifts in social group discussions with rich semantic labels and location data across 17 years of Reddit posts.
- •For public policy teams: Correlate local social attitudes online with policy outcomes to inform targeted interventions and measure impact over time.
Authors
Babak Hemmatian, Sarah Hadjarab, Jessica Chen, Benedek Kurdi
Abstract
We introduce the Illinois Social Attitudes Aggregate Corpus (ISAAC), an open, modular, and accessible corpus of 527 million+ English-language Reddit posts selected for relevance to six key social group distinctions based on race, sexuality, age, ability, body weight, and skin tone, covering the 17-year period from 2007 to 2023. A multi-step, human-audited filtering pipeline was used to keep irrelevant content in the curated dataset below 10%, both overall and for each social group distinction. Each post was then algorithmically annotated with the user's estimated home region, along with a suite of validated off-the-shelf and custom semantic labels including moralization, sentiment, emotion, and linguistic generalization. We confirm the validity of the resulting corpus through convergent evidence linking ISAAC to macro-level societal trends, such as online search behavior, temporal spikes during major societal events (both nationally and regionally), and long-term shifts in public attitudes. By offering a unified, public infrastructure, ISAAC eliminates research fragmentation and enables seamless replication while supporting diverse empirical workflows at scale. Specifically, ISAAC allows investigators to perform cross-category comparisons, conduct high-precision tracking of long-term temporal shifts in social group discourse, and map spatial variation onto localized public opinion and policy outcomes. ISAAC's fully public, modular pipeline facilitates easy extension of the corpus to new platforms, languages, and social categories. To accommodate various research needs, ISAAC is accessible both without coding through a point-and-click website and labeler web-apps, and programmatically via an SQL playground, a Python package, and HuggingFace.