Vision language models struggle to interpret bangla cultural memes
BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset
Computation and LanguageComputer Vision and Pattern Recognition
Summary
Memes from the internet often use pictures and words together to make jokes or express ideas that rely on cultural knowledge. The authors created a collection of 3,000 Bangla memes with detailed labels and explanations to help computers understand these cultural jokes better. They tested current AI models and found that while these models can recognize simple things, they have trouble understanding the deeper cultural meanings and humor. This shows that more work is needed to make AI understand memes in different languages and cultures.
What this means in practice
- •For multilingual ai developers: Develop AI systems that better understand Bangla memes by using a culturally rich and annotated Bangla meme dataset for training and evaluation.
- •For social media content moderators: Improve detection and understanding of humor, sarcasm, and offensiveness in Bangla memes to enhance content moderation accuracy on social platforms.
Authors
Md. Sadman Sakib, Zisan Mahmud, Md. Fahim Arefin, Md Fahim
Abstract
Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-rich content remains insufficiently studied. Internet memes present a challenging setting where meaning emerges from implicit interactions between image, overlaid text, sarcasm, and shared socio-cultural knowledge rather than literal visual recognition. This challenge is amplified in low-resource languages such as Bangla, where code-mixing, stylized scripts, and culturally specific symbolism introduce substantial distribution shift. In this work, we introduce BanglaMemeX, a culturally grounded multimodal benchmark comprising 3,000 Bangla memes annotated with multi-dimensional labels (humor, sarcasm, offensiveness, motivational intent, and overall sentiment) and human-written explanations that explicitly describe textual and visual metaphors. We systematically evaluate modern VLMs on both classification and explanation generation, revealing that current models struggle to interpret implicit cultural cues despite reasonable surface-level accuracy. Our results highlight the need for culturally-aware multimodal systems capable of grounded reasoning under linguistic and cultural distribution shift.