Greek automatic lyric transcription improves with adapted Whisper models
Automatic Lyric Transcription for Greek Songs: Scaling and Task Composition Effects in Whisper Adaptation
Computation and LanguageSound
Summary
Transcribing song lyrics automatically is harder than regular speech recognition because songs have melody, rhythm changes, and background music. This difficulty is even greater for Greek songs, which haven't had much research done before. The authors tested different versions of a model called Whisper to see how well it can transcribe Greek song lyrics. They found that bigger models work better, and training the models with a mix of related tasks helps smaller models. Their best adapted model cut errors significantly, creating the first benchmark for this task in Greek.
What this means in practice
- •For music streaming services: Automatically generate accurate Greek song lyrics to improve music search and user experience.$Commercial implications: Enables lyric display products for Greek music listeners, offering content-enhanced streaming services.
- •For automatic subtitle providers: Provide accurate lyrics for Greek songs to produce subtitles for videos with singing.
Authors
Maria Frangiadaki, Dimitrios Damianos, Kosmas Kritsis, Vassilis Katsouros
Abstract
Automatic Lyric Transcription (ALT) remains substantially more challenging than speech recognition due to melodic variability, rhythmic irregularity, and accompaniment interference. This is heightened in low-resource languages like Greek, where no prior benchmark for ALT exists. We present the first controlled study of Whisper adaptation for Greek ALT, investigating model scaling effects, task composition via multitask training in transcribe-translate ratios, and two-stage speech-to-singing adaptation. We also curate a segment-level aligned singing dataset based on the Greek Audio Dataset (GAD) using source separation and CTC forced alignment. Results show that scaling consistently improves performance, while multitask learning acts as a beneficial regularizer primarily for smaller-capacity models. The 2-stage adaptation in Whisper Large-v3 achieves a Word Error Rate (WER) of 27.2%, a significant improvement over zero-shot baselines, establishing the first Greek ALT benchmark.