SyRHM detects harmful memes using language and reasoning tools
SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detection
Computation and Language
Summary
Harmful memes online can be tricky to spot because they often rely on hidden meanings or cultural ideas. The authors developed SyRHM, a method that finds related memes and breaks down image and text content into symbols and steps to reason about intent. This approach helps the system understand harmful content better than other methods and shows its reasoning along the way. SyRHM was tested on several meme datasets and performed well in identifying harmful messages.
What this means in practice
- •For content moderation teams: Enhance harmful meme detection systems with symbolic reasoning and related meme retrieval to improve accuracy and transparency.
- •For social media platform developers: Integrate a reasoning-based framework to identify and explain harmful content in memes, supporting safer online communities.$Commercial implications: Enables enhanced detection tools for platforms to reduce harmful content effectively and transparently, offering a new product advantage.
Authors
Hanling Wang, Chenlong Wei, Yingjuan Li, Di Wu, Yuchao Zhang, Xiaohui Zhu, Yao Zhu
Abstract
Detecting harmful memes is critical for maintaining safe online communities. However, harmful intent is often implicit, arising from visual-textual incongruity and cultural stereotypes, which challenges existing multimodal detectors. We propose SyRHM, a framework that decomposes harmful meme detection into meaning-grounded retrieval and symbolic-language-enhanced multi-stage reasoning. SyRHM retrieves semantically related memes by parsing multimodal content into textual elements and descriptions, providing grounded context beyond surface-level similarity. Building on the retrieved context, SyRHM uses a translator stage to convert multimodal inputs into symbolic intermediate representations, and then performs multi-stage reasoning via planner and solver stages, enabling expressive and interpretable analysis of harmful intent. Experiments on FHM, HarM, and MultiOff demonstrate the effectiveness of SyRHM, achieving superior performance on most evaluation settings against multimodal and reasoning-based baselines, while providing reasoning traces for harmful content. The code is available at: https://github.com/Scabbards1500/SyRHM