Acoustic plans help find harmful sounds in text to audio models
AUTEUR: Acoustic Plan Learning for Red-Teaming Text-to-Audio Models
Artificial IntelligenceSound
Summary
It can be hard to check if text-to-audio systems produce harmful sounds because text filters won’t catch everything. The authors create a way to describe sounds as a plan, including main sounds, background, and voice features, then use audio feedback to improve these plans. Their method tests models by making sounds that sneak past text checks but expose harmful audio. This approach found flaws in several systems and worked across different models, showing some common weaknesses.
What this means in practice
- •For security teams in ai companies: Automate generating test sounds that bypass text filters but reveal audio risks in text-to-audio products.
- •For audio software developers: Create tools that revise audio prompts to probe and improve safety measures in audio generation systems.
Authors
Jiashu Zhang, Jiaming He, Zheming Zhang, Kaizheng Wang, Bing Li, Zhiyuan Zha, Yi Yu
Abstract
Automated red-teaming of text-to-audio models requires test prompts that pass textual screening and expose harmful acoustic content. The central challenge is to connect the sounds produced by the target model to both reusable test generation and prompt revision. Our key idea is to represent test descriptions as structured acoustic plans that organize foreground sounds, context, and vocal attributes. To this end, we propose acoustic plan learning, a framework that learns and refines these plans using waveform feedback. We train the plan generator through supervised fine-tuning followed by reinforcement learning with audio feedback while keeping the target audio model fixed. At inference time, the same trained model generates initial plans and uses feedback to revise their context and vocal attributes while preserving the foreground. Experiments across four target systems show attack success rates of 79.4% on TangoFlux and 95.6% on Kling. The resulting test prompts also transfer across open-source and closed-source models, indicating shared acoustic vulnerabilities.