Multimodal AI tool tracks origins and risks of unsafe memes
MemeTrace: A Multimodal LLM-Based Agent for Safety-Aware Meme Provenance Analysis
Artificial IntelligenceComputation and LanguageComputers and Society
Summary
Online memes can spread false or harmful content quickly, making it hard for people to know where they came from or how risky they might be. The authors created MemeTrace, an AI tool that looks at both images and text to trace where a meme started, how it spreads, and if it contains dangerous information. They tested MemeTrace on a real collection of memes and found it was better at spotting unsafe memes and could handle image changes without losing accuracy. This helps make it easier to understand and manage problematic memes online.
What this means in practice
- •For content moderation teams: Automatically identify and trace risky memes to improve online safety enforcement.
- •For social media platform engineers: Integrate multimodal analysis tools to enhance detection of harmful meme content under visual changes.
Authors
Bo Shao, Yihan Ma, Yiting Qu, Michael Backes, Yang Zhang
Abstract
Online memes are often carriers of unsafe content such as disinformation and propaganda. Their rapid evolution in response to news events and strong dependence on contextual information make it difficult for platforms and moderators to detect and moderate them effectively. Moreover, existing moderation studies typically classify memes as safe or unsafe based on individual images, lacking the ability to perform broader provenance analysis, i.e., where a meme originates, how it spreads, what cultural symbolism it represents, whether it poses potential risks to society, etc. To bridge this gap, we introduce MemeTrace, a multimodal LLM-based agent that automatically traces the provenance and potential risks of emerging memes. We evaluate MemeTrace on Meme100, a dataset of 100 real-world memes, using both human annotation and automatic metrics. The results demonstrate that MemeTrace substantially outperforms the VLM baseline. More importantly, MemeTrace achieves higher accuracy in detecting unsafe memes, especially those involving disinformation and sensitive content, and remains robust under visual perturbations.