Bangla dialect benchmark aligns transliteration with multiple annotations
5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs
Computation and Language
Summary
Bangla is a widely spoken language with many regional dialects, but most language technology focuses only on the standard version. The researchers created a new dataset called 5-Dialects-BN that includes texts from five Bangla dialects, along with their Romanized versions, English and standard Bangla translations, and labels indicating if the text is subjective or objective. This resource helps improve and test language models on these diverse dialects, supporting tasks like dialect recognition and translation. It is verified by native speakers to ensure accuracy and dialect authenticity.
What this means in practice
- •For natural language engineers: Develop and evaluate language models that can understand and translate diverse Bangla dialects using a multi-annotated benchmark.
- •For social media platform developers: Automatically detect and normalize regional Bangla dialects in user posts to improve content moderation and personalized experiences.
Authors
Md Mahir Jawad, Galib Mahmud Jim, Rafid Ahmed, Mir Sazzat Hossain, Md Fahim, Md Farhad Alam Bhuiyan
Abstract
Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply for low-resource languages and dialectally diverse settings. Bangla, the world's sixth most spoken language, exemplifies this gap: existing resources overwhelmingly target Standard Bangla, leaving its regional dialects without the benchmarks needed to develop or evaluate dialect-aware systems. We address this gap with 5-Dialects-BN, the first multi-annotation Bangla dialect benchmark to align Romanized transliteration with dialectal text, Standard Bangla, English, and subjectivity labels across five regional varieties. The dataset comprises 6,000 manually annotated entries spanning five major dialects: Chittagong, Barisal, Noakhali, Sylhet, and Rangpur (Chittagong 1,900; Noakhali 1,500; Sylhet 1,200; Barisal 700; Rangpur 700), reflecting natural online availability. Each entry is enriched with five aligned annotations: the original dialectal text, a Romanized transliteration, an English translation, a Standard Bangla translation, and a subjectivity label (subjective vs. objective). Annotations were produced and cross-validated by native speakers and undergraduate linguistics students to ensure dialectal authenticity and semantic fidelity. The resulting resource supports a diverse suite of tasks, including dialect identification, dialect-to-standard normalization, machine translation, subjectivity classification, and parameter-efficient fine-tuning (e.g., LoRA) of multilingual LLMs. By providing a standardized, multi-annotation benchmark, 5-Dialects-BN enables principled evaluation of LLMs on dialectally diverse Bangla and lays a foundation for further research in low-resource, dialect-aware NLP.