Persian hate speech analysis improves with multi-step reasoning tasks
From Triage to Impact: A Multi-Hop Reasoning Benchmark for Explainable Persian Hate Speech
Computation and Language
Summary
Hate speech is a growing problem on social media, especially in less-studied languages like Persian. The authors created a new test to better understand and explain hate speech in Persian, breaking it down into detailed tasks like identifying who is targeted and how harmful the speech is. They tested advanced language models and found they struggle more as tasks become more detailed and require reasoning. Adding steps that show the model's thought process helped make severity predictions more accurate. This work can help improve tools that manage hate speech in Persian online communities.
What this means in practice
- •For social media moderators: Use fine-grained, explainable models for detecting and assessing Persian hate speech severity for better content management.
- •For natural language processing engineers: Develop multi-step reasoning components to enhance Persian hate speech detection systems with explainable intermediate reasoning.
Authors
Ghazal Kalhor, Sima Kamandloo, Mehrsa Shamsian, Behnam Bahrak
Abstract
Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relatively low-resource language, lacks sufficient annotated data and computational resources for fine-grained hate speech analysis. In this study, we introduce a fine-grained, multi-hop benchmark for explainable Persian hate speech analysis, covering content triage, target identification, hate tactic and moral foundation analysis, textual evidence extraction, harmful narrative inference, and severity assessment. By evaluating state-of-the-art large language models (LLMs), we find that performance generally declines as tasks require more fine-grained semantic and inferential reasoning. Proprietary models generally outperform open-weight models, while reasoning-oriented models show no consistent advantage. Incorporating intermediate reasoning steps substantially improves severity prediction compared with one-shot assessment. Our analysis further reveals that Incitement to Violence or Harm is most strongly associated with higher severity, while political-affiliation targets are particularly associated with subversion-based moral framing. Overall, this work provides insights into LLM-based hate speech analysis and supports more accountable moderation of Persian-language social media. The benchmark is publicly available at https://github.com/kalhorghazal/From-Triage-to-Impact.