Large language model extracts political claims from social media at scale
Inductive Claims Extraction at Scale
Computation and LanguageComputers and SocietySocial and Information Networks
Summary
People often make short statements about facts or opinions on social media during big events like elections or sports tournaments. These statements, called claims, often appear in patterns linked to different viewpoints. The authors created a method that uses artificial intelligence to find and organize these claims from large amounts of Twitter data. They checked carefully to see how well their method works and showed examples from political and sports discussions.
What this means in practice
- •For political analysts: Identify and track recurring claims in social media to map political divisions and echo chambers during key elections.
- •For social media monitoring teams: Automatically extract and catalogue claims from Twitter to analyze public opinion trends around major events like sports championships.
Authors
Sandrine Chausson, Björn Ross
Abstract
A large part of political discourse on social media is built and expressed at a level of claims: i.e. declarative, typically single-clause statements, which convey a particular interpretation of reality and can range from factual to evaluative. Moreover, rather than occurring randomly, claims coalesce, recur in patterns, and come to be associated with different world views. When paired with structural computational tools such as Social Network Analysis, claims can be a powerful unit of analysis to study political phenomena such as echo chambers or polarisation. In this paper, we present a pipeline that uses a large language model (LLM) to inductively extract and catalogue claims from large social media corpora, and apply it to two different Twitter datasets: one relating to the 2020 US presidential election and the other to the 2022 FIFA World Cup. We comprehensively evaluate the approach by measuring the pipeline's recall and precision against manually annotated samples, run ablation studies isolating the contribution of its various components, and perform a qualitative error analysis. We discuss the value of the approach in the context of Computational Social Science research, and illustrate its capabilities by presenting the claims catalogue obtained from each dataset.