Polish Silesian translation system beats open source and commercial models
Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Models
Computation and Language
Summary
Translating texts between dialects is hard because they vary a lot and there isn’t much data to learn from. The authors worked on translating between Polish and the Silesian dialect and made a special test set to check translation quality. They found that their rule-based translation system works better than advanced neural methods and commercial systems when handling dialect-specific text. They also improved a neural model with careful training but it still didn’t beat the rule-based approach on tricky dialect data.
What this means in practice
- •For language technology developers: Build dialect-aware translation tools for Polish-Silesian text using the provided test data and models to improve accuracy beyond generic systems.
- •For machine translation engineers: Enhance neural machine translation systems by fine-tuning them on carefully curated dialect-specific datasets to improve dialect translation performance.
Authors
Grzegorz Kulik, Mikołaj Pokrywka, Adam Jatowt, Wojciech Kusa
Abstract
Dialectal machine translation remains challenging due to limited data and strong linguistic variation not captured by standard benchmarks, which often assume standardized and well-edited text. We study Polish-Silesian MT using neural and rule-based systems, evaluating on SiLTT - a new Pol-Szl testset, alongside established BOUQuET and FLORES benchmarks. Results show our rule-based system is consistently strongest on SiLTT and BOUQuET datasets and that TranslateGemma fine-tuned on a curated dataset improves over strong neural baselines but does not surpass the rule-based system in dialectal settings. We release SiLTT and our best neural model to support further research.