Author context improves political stance detection accuracy on social media
Author-Informed Stance Detection: A Dataset and Findings
Computation and Language
Summary
Figuring out someone's political opinion from a short social media post is hard, especially without knowing about the person writing it. The authors found that both humans and AI models are better at guessing political views when they have background info about the author. This extra context helps a lot for tricky posts like sarcastic tweets. They also created a big dataset with tweets and author info to help others build better stance-detection tools.
What this means in practice
- •For social media monitoring teams: Detect user political opinions more accurately in tweets by incorporating author background information.
- •For digital marketing teams: Improve targeting of political campaigns using stance detection models enhanced with author-aware training.
Authors
Ogen Schlachet Drukerman, Einat Minkov
Abstract
Stance detection in social media is challenging, as interpreting a text often requires understanding the author's perspective. We re-frame political stance detection as an author-informed text understanding task, where it is desired to infer both the author's political leaning and their stance towards the target from the text. We first show that human annotators struggle to infer authors' political leanings and stances from individual tweets, and that pretrained large language models (LLMs) perform below human level in this setting. We then demonstrate that author context improves stance detection for both humans and LLMs, with particularly pronounced benefits for tweets perceived as sarcastic. Through extensive experiments, we further show that LLMs can be effectively fine-tuned to predict authors' political affiliations, and that author-informed fine-tuning yields better stance detection performance than standard fine-tuning. This approach not only surpasses human performance but also improves generalization across datasets. To support further research on author-informed political stance detection, we release a large dataset of stance-labeled tweets enriched with author-context information.