LLMs show limited success in judging counterspeech quality
Evaluating LLMs as Judges of Counterspeech Quality Against Online Hate Speech
Computation and Language
Summary
Hate speech online can harm people, and counterspeech is a way to respond without censorship. The authors studied if large language models (LLMs) can judge how good counterspeech is, to reduce the need for humans who get tired or hurt by this work. They found that LLMs often disagree with human experts and with each other to some extent, though some models do better at certain evaluations. This means using LLMs to judge counterspeech quality is promising but still has important challenges.
What this means in practice
- •For social media moderation teams: Use LLMs to support human review by pre-assessing counterspeech quality and filtering content that needs expert attention.
- •For content platform trust and safety engineers: Integrate LLM-based counterspeech evaluation tools to reduce emotional strain on human annotators managing large volumes of online hate speech.
Authors
Johnny Alali, Nicolás Benjamín Ocampo, Davide Ceolin
Abstract
Prolonged exposure to hate speech on social media can cause lasting psychological distress among users. Counterspeech has emerged as an effective, non-censorial mitigation strategy, but its deployment requires rigorous quality assessment. Manual assessment is unscalable due to the large volume of content on the Web and imposes severe emotional strain on human annotators. Large Language Models (LLMs) offer a scalable alternative, yet concerns remain regarding their reliability. In this paper, we investigate the extent to which LLMs can effectively perform this task. We test 12 open-weight LLMs of different sizes and families under zero-shot, few-shot, and chain-of-thought prompting, to approximate pointwise and pairwise human expert assessment of counterspeech quality across four dimensions: relevance, coherence, aggressiveness, and suitableness. We find that (i) LLMs show low agreement and moderate rank correlations with human experts: our best configuration, Qwen 2.5 72B on coherence, achieves up to a linear-weighted Cohen's $κ_l = 0.19$ and Spearman's $ρ= 0.42$ in pointwise evaluation and an unweighted Cohen's $κ= 0.25$ in pairwise evaluation. (ii) Converting pointwise scores into pairwise labels substantially improves human--LLM agreement for selected configurations. For same-instance coherence, Qwen 2.5 72B under zero-shot prompting achieves $κ= 0.35$ with converted labels, compared with $κ= 0.25$ when judging pairs directly, and Mixtral 8$\times$7B under few-shot prompting improves from $κ= 0.07$ to $κ= 0.34$. (iii) LLMs agree more with one another than with human experts, particularly in pairwise evaluation, where the strongest model pair, OLMo 2 32B and Llama 3.3 70B on coherence, reaches substantial agreement with $κ= 0.76$, whereas their agreement with the experts in the same configuration does not exceed $κ= 0.21$.