Security risks in audio ai agents from sound input to actions
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
Cryptography and SecurityComputer Vision and Pattern RecognitionMachine Learning
Summary
Audio AI systems like those in smart homes and cars listen to sounds and then act on them, which can create security risks. The authors identify gaps in how these systems are protected, especially as sound-based commands move through different parts of the system. Their survey looks closely at how security problems can start from the sound input and lead to unwanted actions, highlighting that defenses at one stage don’t always keep the whole system safe. This helps show the need for thorough security checks that cover the entire process, from hearing sounds to acting on them.
What this means in practice
- •For smart home security teams: Identify security gaps in smart home voice agents by tracing attack paths from microphones to device actions.
- •For automotive system engineers: Evaluate how audio command inputs on vehicles might be exploited to cause unintended behavior across embedded systems.
A survey. It maps existing work.
Authors
Binhong Wang, Haidong Kang, Yanfeng Zhang, Hong-Ning Dai, Guo Yu, Shangce Gao
Abstract
Audio Large Language Models (Audio LLMs) are embedded in agents for smart homes, vehicles, wearables, and embodied systems, where acoustic inputs can propagate from perception into persistent state and consequential execution. This setting extends security analysis beyond model behavior to the system conditions that determine how acoustic inputs can influence agent behavior and external actions. Existing surveys examine acoustic security, LLM-based agents, and edge deployment in isolation, leaving their security interactions in deployed Audio LLM-based agents insufficiently characterized. This survey introduces an Acoustic Sensing-to-Action (S2A) system model that traces these security implications along the path from acoustic sensing to external execution, while accounting for edge-cloud runtime and lifecycle transformation. We use the model to synthesize attacks, defenses, and evaluation methods across the acoustic, agent, and edge-system literature. The synthesis shows that security gaps can emerge within components and at the transitions between stages, where protections developed for one stage do not necessarily provide end-to-end assurance in the deployed system. These findings motivate security evaluation that follows acoustic influence from sensing to action across persistent state, system boundaries, and changing deployment conditions in real-world agents.