Transformer system completes Bengali sentences for dementia patients
Automatic Sentence Completion Using Bengali Audio Speech with a Transformer-Based Early Speech Recognition System
Computation and Language
Summary
Dementia makes it hard for people to finish their sentences and communicate clearly. The authors created a tool that listens to Bengali speech and detects signs of dementia by analyzing voice and text features. Then, it helps complete sentences the person struggles to finish, using a special computer model called a Transformer. This system aims to make talking easier for people with dementia and supports caregivers in understanding them.
What this means in practice
- •For speech technology developers: Create speech-to-text systems that identify cognitive decline and assist in sentence completion for Bengali speakers with dementia.
- •For healthcare communication tool designers: Integrate auto-completion features powered by Bi-LSTM models to help caregivers communicate effectively with Bengali dementia patients.$Commercial implications: Enables development of assistive communication devices that improve interaction between caregivers and dementia patients in Bengali-speaking communities.
Authors
Md Asif Akram, Arif Faisal, Sakir Hossain Faruque, Sharun Akter Khushbu
Abstract
Dementia severely impairs communication abilities and turns everyday conversations into challenging experiences. Individuals affected by dementia frequently struggle to express their thoughts or complete sentences, making it difficult for family members and caregivers to understand their needs. This study proposes a novel approach to assist dementia patients by developing an integrated system that detects signs of dementia through Bangla speech and text analysis and provides sentence auto-completion support during conversations. To achieve this, recorded voices from individuals with and without dementia were analysed, and key acoustic features were extracted to capture vocal patterns associated with cognitive decline. A transformer-based model was employed to transcribe voice signals into text for linguistic analysis. In the classification task, several Machine Learning (ML) and Deep Learning (DL) models were evaluated using both acoustic and textual representations, and the LSTM model architectures demonstrated strong capabilities for modelling contextual patterns in speech and language data. For the sentence auto-completion task, a sequential learning model based on a Bidirectional LSTM (Bi-LSTM) network with an attention mechanism was developed to sequentially complete partially expressed Bangla sentences during conversation. This research bridges the gap in dementia analysis in an underrepresented language and implements a sentence auto-completion framework for cognitive disorders.