Hate speech detection models miss many criminal posts and overflag others
MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short
Computation and Language
Summary
Detecting hateful and illegal posts online is important but challenging. The authors tested different ways to help AI models recognize criminal hate speech by showing them similar examples before making decisions. They found that while some examples help a bit, the choice of AI model is much more important. The models often flag harmless posts as criminal and still miss many truly criminal ones, making them better suited to assist human reviewers rather than automatically removing posts.
What this means in practice
- •For social media moderation teams: Use retrieval-based in-context learning to better triage potentially illegal hate speech posts for human review.
- •For legal tech developers: Incorporate legal knowledge into AI models to improve identification of defamatory offences in social media content.
Authors
Kristin Gnadt, Maximilian Meidinger, Matthias Aßenmacher
Abstract
With hate speech being ubiquitous online, automatic detection is crucial, in particular when it comes to criminally relevant social media posts. We study a variety of retrieval-based in-context learning (RetICL) strategies for detecting defamatory offences under §§ 185-187 StGB (the subject of GermEval 2026 Subtask 4). Few-shot prompting beats zero-shot, but retrieval-based approaches offer only marginal gains over random demonstrations, and even fall behind an optimised static set of demonstrations. Providing concrete legal knowledge helps, yet model choice outweighs every other system choice. Models over-predict criminal relevance while still missing 26-57% of criminally relevant posts, suiting them for triage rather than autonomous moderation.