Smart home assistant remembers and retrieves household conversations
Your Home Remembers: A Voice-Enabled Household Memory Assistant for Conversational Retrieval
Human-Computer Interaction
Summary
Household conversations often contain important details that get forgotten because they are never written down. The authors created a voice-powered assistant that listens to these talks, saves them, and can answer questions about what was said. It works on an Android tablet and uses different speech and language tools that can be swapped in or out. When you ask something, the assistant finds relevant parts of past conversations to give an answer. You can also check when and where the information came from.
What this means in practice
- •For smart home developers: Build voice assistants that remember household conversations and answer related questions automatically.$Commercial implications: Enables development of memory-focused smart home products that provide conversational retrieval of household information.
- •For speech software engineers: Integrate modular speech recognition and diarisation components into voice-controlled applications using its orchestration design.
Authors
Aleix Sant, Jordi Luque, Fernando López, Pablo Gómez, Carlos Escolano
Abstract
Everyday household information is often exchanged in conversations but never written down, making details easy to forget or misremember. We present a voice-enabled smart-home assistant that acts as a household memory by recording conversations and retrieving information from stored and newly captured speech. The assistant runs on an Android tablet with a modular architecture in which speech-recognition, diarisation, embedding and language models can be configured or replaced through an orchestration layer. On-device wake-word detection and VAD-based query capture support hands-free interaction. Recorded audio is processed in configurable chunks, transcribed and diarised, converted into turn-aware passages with conversation-level timestamps, and indexed in a vector store. For memory queries, a retrieval-augmented generation pipeline selects key passages and uses them to generate an answer. The interface supports recording, conversation management and inspection of timestamped sources.