Cal mos improves speech quality ratings by adapting model layers
CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models
SoundArtificial Intelligence
Summary
Voice quality matters for things like phone calls and voice assistants, but computers need to judge it correctly. The authors studied many speech AI models to find which parts of these models best predict human ratings of voice quality. They found that it depends on the model and the data, and just combining layers can be unreliable. They created a way to adjust each layer before combining them, making predictions more stable and closer to full retraining but without changing the entire model.
What this means in practice
- •For speech technology developers: Improve automatic voice quality evaluation tools by using layer-adapter fusion without retraining the whole speech model.
- •For telecommunications engineers: Enhance call quality monitoring by incorporating robust multi-layer speech model features calibrated per dataset and model.
Authors
Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva, Daniel Casanova, Rafael Faustino, Frederico Oliveira, Arlindo Galvão Filho, Anderson da Silva Soares
Abstract
Speech Quality Assessment (SQA) is essential for modern speech technologies, and recent non-intrusive SQA predictors increasingly rely on Speech Foundation Models (SFMs). However, because SFMs expose representations from many layers, it remains unclear which depths are most informative for MOS prediction and how multi-layer information should be combined reliably across backbones and datasets. We benchmark ten SFMs on four MOS datasets under three regimes: full fine-tuning, last-layer probing with a frozen encoder, and naive cross-layer weighted aggregation. We find that the best layer is strongly backbone- and dataset-dependent, and that naive weighted fusion can be unstable across settings. We further evaluate a layer-calibrated aggregation variant that applies per-layer adapters before pooling, which improves the robustness of multi-layer fusion and narrows the gap to full fine-tuning while keeping the backbone frozen.