Decision models show varied success on Vietnamese text classification
How Well Do Decision Models Transfer to Vietnamese? An Empirical Study of Text Classification
Computation and Language
Summary
Classifying text without training examples is tricky, especially in Vietnamese. The authors tested four decision-making models on different kinds of Vietnamese text tasks to see how well they work. They found one model, JEV, often performs best using English label descriptions. They also discovered that removing Vietnamese special marks (diacritics) greatly reduces accuracy, more than slang or typos. Choosing how to describe categories and handling Vietnamese writing variations are important for good results.
What this means in practice
- •For natural language processing teams: Design Vietnamese text classifiers that account for diacritic marks to maintain accuracy in zero-shot settings.
- •For social media platform developers: Improve content filtering for Vietnamese texts by addressing common slang and typographical variations without losing precision.
Authors
Phat Huu Ha, Nguyen Xuan Phuc, Nong Nhut Quy, Trinh Tran Tran, Nguyen Minh Triet, Dang Van Thin
Abstract
Recent advances in specialized decision models have expanded the possibilities for zero-shot classification, yet their effectiveness on Vietnamese remains underexplored. This study evaluates four specialized decision models across 13 classification settings spanning six task groups, examining zero-shot performance, sensitivity to label-description language, and robustness to Vietnamese linguistic variations. The evaluation considers English, Vietnamese, and bilingual label descriptions, alongside common linguistic perturbations involving teencode, typographical errors, and diacritic removal. Under English label descriptions, JEV achieves the strongest overall performance, ranking first in 11 classification settings. The effects of label-description language vary across models and datasets, with bilingual descriptions providing no consistent improvement. Robustness experiments reveal that diacritic removal causes the most substantial performance degradation, reducing mean macro-$F_1$ by 9.51--15.47 points across model configurations, considerably more than teencode or typographical errors. These findings highlight the importance of label-description choices and Vietnamese orthographic variation when evaluating specialized decision models for zero-shot text classification.