Thaka at KSAA-2026 Task 2: Regularized Fine-Tuning for Arabic Speech Diacritization
2026-05-25 • Computation and Language
Computation and LanguageSound
AI summaryⓘ
The authors developed a system that turns spoken Arabic into fully vowel-marked text, even when only a small amount of training data is available and no extra data can be used. Their method fine-tunes a multimodal model that combines speech and text understanding, and they improve learning stability using special regularization techniques. During prediction, they average many model outputs to get better results. Their system performed best in the KSAA-2026 Shared Task, achieving the lowest word error rate for this challenge.
Arabic diacritizationspeech recognitionCATT-WhisperR-Drop regularizationOptuna hyperparameter tuningFocal LossMonte Carlo Dropoutword error ratemultimodal modelshared task
Authors
Meshal Alamr, Hassan Alqaeri, Abdullah Aldahlawi
Abstract
We describe the winning system for Task 2 of the KSAA-2026 Shared Task on Arabic Speech Dictation with Automatic Diacritization. The task requires producing fully diacritized Arabic text from speech audio and undiacritized transcripts, with only 2,327 training samples available and no external data permitted. Our system fine-tunes CATT-Whisper, a character-level multimodal model combining a pretrained CATT text encoder with a frozen Whisper speech encoder. The key to our approach is training regularization: R-Drop consistency regularization, Optuna-optimized hyperparameters with high weight decay, and Focal Loss. At inference, we average 200 stochastic forward passes across four model checkpoints using Monte Carlo Dropout at the softmax probability level. The system achieves 23.26% WER on the primary leaderboard metric (with case endings, including no-diacritic positions), placing 1st among all participants.