Cognitive instability signals early decline in verbal fluency speech
CCMAN: Cognitive Instability-Aware Cross-Modal Attention Network for Interpretable Temporal Biomarkers of Verbal Fluency Speech
SoundMachine Learning
Summary
Detecting early signs of thinking problems from how people talk could help find these issues sooner without clinical tests. The authors created a smart system called CCMAN that listens to different parts of speech—meaning, sounds, and language patterns—over time during word-generation tasks. It finds subtle time-based changes, like varying pauses and shifts in meaning, that link to mild cognitive decline and dementia. Tests with hundreds of people show this method works better than others and can be used on new speech data.
What this means in practice
- •For healthcare data teams: Integrate speech-based temporal biomarkers into early screening tools for cognitive impairment using brief verbal fluency recordings.
- •For smart device developers: Implement adaptive systems that monitor cognitive health via speech by analyzing temporal speech instability in everyday interactions.$Commercial implications: Enables cognitive monitoring features to be sold in consumer health devices and apps with improved early detection capabilities.
Authors
Madhurananda Pahar, Caitlin Illingworth, Dorota Braun, Daniel Blackburn, Heidi Christensen
Abstract
Early detection of cognitive decline from speech offers a scalable and non-invasive alternative to conventional clinical assessment. Verbal fluency tasks are particularly informative, but most automated approaches aggregate features across an entire recording, overlooking temporal speech dynamics. We propose the Cognitive Instability-Aware Cross-Modal Attention Network (CCMAN), a transfer learning framework that learns task-agnostic cognitive speech representations from multiple memory-probing tasks before fine-tuning on a minute-long semantic and phonemic verbal fluency task. CCMAN integrates semantic, acoustic, and linguistic information through bidirectional cross-attention, gated multimodal fusion, and transformer-based temporal modelling to derive interpretable biomarkers of cognitive decline. Experiments were conducted on 165.44 hours of speech from 843 participants (498 healthy controls, 245 with mild cognitive impairment, and 100 with dementia). CCMAN achieved Macro-F1 scores of 0.81 and 0.59 for binary and multiclass semantic fluency classification, and 0.77 and 0.53 for phonemic fluency, consistently outperforming strong static and temporal baselines. Statistical analyses showed that semantic drift variance and pause variance, but not mean semantic drift, were significantly elevated in both MCI and dementia relative to healthy controls, while pause duration increased progressively over the task with the steepest slope in dementia, supporting global and progressive temporal speech instability as interpretable biomarkers. Evaluation on the independent PROCESS-2 benchmark further demonstrated the generalisability of the proposed framework, improving the baseline Macro-F1 by up to 9%. These findings support temporal speech instability as a dynamic speech biomarker for robust, interpretable, and generalisable early detection of cognitive decline.