Dual task merging improves low resource alzheimer disease speech detection
Dual Task Vector Merging: Leveraging Emotion and ASR Knowledge for Low-Resource Alzheimer's Disease Detection
Computation and LanguageSound
Summary
Detecting Alzheimer's Disease from speech is hard when there isn't much data to learn from. The authors found a way to help a powerful speech model use knowledge from related tasks like recognizing emotion and speech words. They combined this extra knowledge carefully, so they only needed to adjust a tiny part of the model for Alzheimer's detection. This approach worked better than simply training the whole model or locking it in place, especially when data was scarce.
What this means in practice
- •For speech technology teams: Build Alzheimer's detection models that work better with limited training speech data by merging knowledge from emotion and speech recognition tasks.
- •For clinical ai developers: Integrate dual task vector merging to efficiently adapt speech models for multi-stage cognitive impairment classification using minimal additional training.
Authors
Yuan-Chao Li, Yin-Long Liu, Liu He, Sizhe Liu, Ji-Xuan Deng, Yin Zhang, Zhen-Hua Ling, Jiahong Yuan
Abstract
Alzheimer's Disease (AD) detection from speech benefits from both paralinguistic and linguistic cues, yet model training is frequently constrained by limited data. Fine-tuning a large pretrained speech encoder can easily overfit, while a frozen probe leaves the encoder's task-relevant knowledge underutilized. Therefore, we propose Dual Task Vector Merging, a parameter-efficient method that adapts a frozen wav2vec 2.0 backbone using task knowledge learned outside the AD task. Specifically, we inject two task vectors, derived as weight deltas from emotion- and ASR-fine-tuned wav2vec 2.0 models, into each transformer layer. During AD training, only layer-wise merging coefficients and a linear classification head are optimized, updating less than 0.1% of the parameters required for full fine-tuning. Speaker-independent experiments on ADReSS and ADReSSo show that dual merging improves over frozen probes, LoRA, and full fine-tuning, in the main binary settings, and gives the best results when combined with LoRA. Evaluation on PROCESS further shows that our advantages transfer to three-class HC/MCI/dementia classification when paired with LoRA. Experiments under reduced training data (20%, 40%, and 80%) confirm reliable gains in low-resource settings.