MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors created a new test called MMDS-Bench to see how well computers can understand if a reply agrees, disagrees, or relates in some way to a parent message on social media, especially when images and memes are involved. They included thousands of examples with detailed labels and challenges like understanding jokes or complex reasoning. They checked many advanced language models and found that these models still have trouble understanding the relationship between messages when images and tricky reasoning are needed. The authors also introduced a method to better judge how well models reason through these tasks.
dynamic stance classificationmultimodal learninglarge language modelssocial media analysisparent-reply interactionsstance detectionstructured reasoningmultimodal fusionmemesLLM evaluation
Authors
Yuzhe Ding, Kang He, Li Zheng, Shengwu Zheng, Teng Shi, Fei Li, Chong Teng, Donghong Ji
Abstract
Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work has mainly studied this problem in text-only settings, while social media interactions increasingly rely on images, screenshots, memes, reaction images, and cross-modal references. We introduce MMDS-Bench, a diagnostic benchmark for multimodal dynamic stance classification in social media parent-reply interactions. MMDS-Bench contains 3,482 multimodal instances annotated with a seven-label dynamic stance taxonomy, together with an 800-instance diagnostic subset that requires structured reasoning over parent understanding, reply understanding, and stance-relation inference. We further annotate each instance with five challenge factors covering multimodal fusion, parent framing, non-literal expression, interaction reasoning, and label-boundary ambiguity. We evaluate 12 closed-source and open-source multimodal large language models and propose a reference-grounded LLM-judge protocol for assessing reasoning quality. Results show that current MLLMs still struggle with multimodal dynamic stance understanding, especially in cases that require relational inference beyond separate parent and reply comprehension.