Bangla crisis sentiment shifts tracked with new large social dataset
Can LLMs Follow the Pulse of a Crisis? Evaluating Crisis Sentiment in Bangladesh's July Uprising
Computation and Language
Summary
Understanding public feelings during a crisis is hard, especially in languages like Bangla that don't have many tools. The authors created a big collection of about 200,000 social media comments from the Bangladesh uprising in July-August 2024, including reactions over several key moments. They showed that looking at the bigger conversation helps computers understand emotions better, but machines still get confused by sarcasm and hidden political meanings. This dataset helps test and improve tools that read emotions in fast-changing crisis situations for Bangla.
What this means in practice
- •For social media monitoring teams: Identify shifts in public sentiment during different crisis phases in Bangla-speaking communities using the UNRESTSENT200K dataset.
- •For content moderation teams: Improve detection of nuanced negative sentiment such as sarcasm or implicit political references in Bangla crisis-related comments.
Authors
Md. Samiul Alim, Mahir Shahriar Tamim, Tanvir Ahmed Khan, Sharjil Khan, Rafia Ferdous Duti, Shahriyar Zaman Ridoy, Mohammad Ali Moni
Abstract
Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidly. We introduce UNRESTSENT200K, a Bangla crisis sentiment dataset with approximately 200K Facebook and YouTube comments from the July-August 2024 Bangladesh uprising. The dataset covers five event-aligned phases, from early escalation and internet blackout to regime transition and a later flood crisis. Each comment is linked to its parent post, enabling evaluation with and without discourse context. All comments are annotated through a fully human process involving 14 native Bangla-speaking annotators and senior validation, achieving substantial agreement (kappa = 0.73, alpha = 0.71) and 94.2% blind-audit agreement. We benchmark fine-tuned encoders, prompted LLMs, and LoRA-tuned LLMs. Results show that parent-post context consistently improves performance, while temporal shift across phases causes large performance drops. Strong LLMs perform well, but still struggle with sarcasm, implicit political references, and phase-dependent meaning. UNRESTSENT200K provides a benchmark for studying context-aware and temporally robust sentiment analysis in low-resource crisis discourse. UNRESTSENT200K is available at https://sami0055.github.io/UNRESTSENT200K/