BanglaShield improves safety detection across Bangla scripts and code-mixing
BanglaShield: A Script-Robust Safety Classifier and Benchmark for Bangla Native, Romanized, and Code-Mixed Prompts
Computation and LanguageMachine Learning
Summary
Detecting harmful or offensive text in Bangla is hard because it is written in multiple ways: the native Bangla script, using Latin letters, or mixed with other languages like English. The authors collected over 10,000 Bangla prompts labeled by safety level and created matched sets in all three forms. They tested existing large language models and found these models often miss unsafe content especially when texts are romanized or mixed. They then built BanglaShield, a smaller model tuned on these data, which detects unsafe content much better across all script types.
What this means in practice
- •For chatbot developers: Use BanglaShield to detect and filter unsafe user inputs written in native, romanized, or mixed Bangla scripts for safer conversational AI.
- •For moderation teams: Implement BanglaShield to improve content moderation accuracy across diverse Bangla text forms on social media and online platforms.
Authors
Mohsinul Kabir, Md. Tariquzzaman, Mohammad Rashidujjaman Rifat, Tharindu Ranasinghe, Nishat Raihan
Abstract
Safety classifiers for large language models (LLMs) are typically built and evaluated on standard-script text, yet Bangla, like many lower-resource languages, is routinely written in native script, in Latin-script romanization, and code-mixed with English, Hindi, or Urdu. We introduce BanglaShield, a resource and model for graded prompt-safety classification across these written forms. We consolidate 10,245 Bangla prompts from six sources under a four-level taxonomy (Safe, Offensive, Harmful, Severe) and render each prompt in three aligned forms that share labels and a group-aware split, enabling strictly paired comparisons. Benchmarking six contemporary LLMs, including a Bangla-specialized one, on 36,882 classification queries and 24,318 judged responses, we find that LLM classifiers label 47.0% of unsafe native-script prompts as Safe, rising to 63.4% under romanization, while response-level safety accuracy drops by 10.3 points. Three multilingual guard models also pass 30.3-57.3% of unsafe native-script prompts. Using a model's own classification as an input filter reduces harmful compliance by only about five points. In contrast, BanglaShield, a compact fine-tuned encoder, reaches 87.8 macro-F1 on native-script and 84.4 on romanized prompts, outperforming the best LLM on each form by 42.8-46.1 points with ~130x fewer parameters than Gemma 4 31B. We release the corpus, splits, judgments, and model.