High quality data improves bengali coreference resolution more than data quantity
Quality over Quantity: Rethinking Data and Model Assumptions for Bengali Coreference Resolution
Computation and Language
Summary
Coreference resolution is a language task that finds words referring to the same thing in a text. The authors studied this task for Bengali, a language with few annotated examples. They found that small amounts of well-annotated data work better than large, mixed-quality data. Techniques like data augmentation or transferring from other languages didn’t consistently help and sometimes made results worse. This suggests that for Bengali and similar low-resource languages, it’s better to focus on high-quality annotations rather than collecting lots of lower-quality data.
What this means in practice
- •For natural language processing teams: Improve Bengali coreference systems by prioritizing small high-quality annotated datasets over large noisy ones.
- •For multilingual ai developers: Avoid relying on cross-lingual transfer from English or Hindi when building Bengali coreference resolution models to prevent performance drops.
Authors
Zenith Biswas, Andrew Thomas Dyer
Abstract
Multilingual transformer models are often assumed to generalise across languages, but their effectiveness for coreference resolution in low-resource settings remains unverified. We investigate this gap in Bengali, a language with over 250 million speakers but limited annotated coreference resources. We evaluate transformer-based pre-trained encoders under multiple settings, including zero-shot, fine-tuning, data augmentation, and cross-lingual transfer. Our results challenge several common assumptions about model generalisation and data efficiency in low-resource NLP. We demonstrate that a small amount of high-quality annotated data significantly outperforms larger mixed-quality datasets (p < 0.01). We find that data augmentation, particularly back-translation, does not improve performance and can degrade results in coreference resolution. Cross-lingual transfer is also inconsistent: transfer from a distant language like English reduces performance, and transfer from the related language Hindi provides no consistent benefit across models. Several widely used multilingual models exhibit collapsed embedding spaces, rendering them unsuitable for similarity-based clustering tasks. These findings highlight the importance of annotation quality over data quantity and provide empirical guidance for developing coreference resolution systems in low-resource languages.