Local smart home control improves accuracy with typed decision model
Trail: Accurate Local Smart Home Control Using Decision Models
Artificial IntelligenceMachine Learning
Summary
Controlling smart home devices using natural language can be tricky because big cloud-based AI models raise privacy concerns and smaller local models often make mistakes. The authors created a system called Trail that runs entirely on local devices and makes smart home actions by choosing from a list of valid options like room, device, and operation. This approach reduces errors and allows the system to skip commands it isn't confident about. They also built a memory feature to remember user preferences and previous conversations, improving multi-turn interactions.
What this means in practice
- •For smart home developers: Build smarter on-device voice control systems that ensure valid commands and protect user privacy without cloud access.$Commercial implications: Enables local smart speaker and home automation products that are more accurate and privacy-preserving, suitable for commercialization.
- •For voice assistant engineers: Enhance multi-turn conversational abilities in local voice assistants using structured memory to manage device states and user preferences.
Authors
Maleeha Masood, Deepak Vasisht, Indranil Gupta
Abstract
Natural-language driven smart home control remains challenging: cloud-hosted Large Language Models (LLMs) raise privacy concerns, while local Small Language Models (SLMs) can generate invalid actions or misinterpret requests. We introduce Trail (Typed Registry-Anchored Inference for Language commands in Smart Homes), a fully local framework that replaces open-ended command generation with hierarchical typed decisions. Trail exploits the natural structure of a home's action space, selecting a room, device, and operation from candidates derived from the runtime device registry. This ensures valid actions while enabling confidence-based abstention when requests cannot be reliably resolved. We also design a structured memory module that uses typed decisions to support user preferences, device references, and multi-turn interactions. To evaluate Trail, we extend HomeBench and generate three datasets: Trek-Base, which tests natural-language variation; Trek-Robust, which evaluates robustness to changes in the home and its description; and Trek-Memory, which assesses conversational memory across multi-turn interactions. Trail achieves 85.3% accuracy on Trek-Robust versus 69.8% for a finetuned Qwen3.5-4B baseline, while remaining competitive on Trek-Base requests. With memory, Trail resolves 75.6% of the requests that depend on earlier turns, compared with 42.5% for the finetuned baseline given conversation history.