Smart home agents improved for better failure handling and control
Increasing Resilience of Smart Home Agents
Multiagent Systems
Summary
Smart homes use AI to control devices and manage tasks automatically, but current AI helpers often struggle with tricky or unexpected situations. The authors looked at using large language models (LLMs), a kind of AI that understands and generates text, inside a popular smart home platform called HomeAssistant. They tested different ways to train and guide these AIs to handle hard tasks and recover from mistakes more reliably. Their results show that fine-tuning these models works best for controlling many devices, while newer AI strategies help in dealing with failures.
What this means in practice
- •For smart home developers: Improve AI agents in smart home platforms to better manage multiple devices and recover from failures.
- •For home automation integrators: Deploy more resilient AI control in complex smart home environments to reduce service interruptions.
Authors
Christopher Terrazas, Eduardo Cotilla-Sanchez
Abstract
Smart homes and smart devices are becoming more prevalent across millions of homes around the world. With the rise of AI, the smart home industry is quickly increasing its integration to manage common smart home tasks. However, existing work in large language models (LLMs) as agents within smart homes have shown minimal resilience due to limited environment scenarios or poor performance in complex tasks. We explore several strategies for LLMs as agents within the popular open-source software (OSS) smart home automation framework HomeAssistant to increase overall smart home resilience. Our approach combines traditional supervised learning techniques and optimized prompting as the core learning process. We use a diverse set of LLMs covering different levels of reasoning and costs in our optimization pipeline and evaluate their performance on a subset of the hardest tasks in a smart home benchmark. We include ReAct and Reflexion agentic paradigms and reveal how both provide marginal return on investment compared to fine-tuned LLMs for multi-device control within HomeAssistant but show promise in resilience tasks such as failure response.