A benchmark measures safety risks in text to audio video creation
AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Text-to-audio-video (T2AV) systems create videos with sound and speech from text prompts, but it’s tricky to check if they make unsafe content. The authors made a special benchmark called AV-SafetyBench to test safety across both video and audio together, not separately. They tested five different T2AV models and found many unsafe cases would be missed if only audio or video were checked alone. Their work helps make T2AV safety testing more complete by looking at how sound and visuals combine.
What this means in practice
- •For multimedia ai developers: Check T2AV model outputs for unsafe content across video, speech, and sound to improve safety mechanisms.
- •For content moderation teams: Use AV-SafetyBench to understand risks from combined audio and video in automatically generated media for better policy enforcement.
Authors
Suah Choi, Tae-Young Lee, Gyeong-Moon Park
Abstract
Recent text-to-audio-video (T2AV) models jointly generate video, speech, sound effects, and ambience from a single text prompt. This capability poses new challenges for safety evaluation, as unsafe content may be conveyed through the audio track or arise only when the visual and audio tracks are interpreted jointly. Existing safety benchmarks largely focus on either generated video or generated audio in isolation and are therefore not designed to capture these risks. To close this gap, we introduce AV-SafetyBench, the first safety benchmark developed specifically for T2AV generation. AV-SafetyBench comprises a four-axis, 13-category taxonomy and 5,200 manually reviewed prompts that specify visual scenes, speech, and non-speech audio. Our evaluation protocol assesses each output under three views: Full-AV, Video-Only, and Audio-Only. It then uses the Video-Only and Audio-Only judgments to assign Full-AV unsafe outputs to one of four risk sources: Video-Only, Audio-Only, AV-Both, or AV-Joint. We evaluate five open-source T2AV models and validate the automated Full-AV judgments against human annotations. Across the five models, Full-AV Unsafe Rates range from 25.1% to 49.4%. Beyond these aggregate rates, risk-source analysis reveals that, for four of the five models, Audio-Only and AV-Joint cases-unsafe outputs missed by video-only evaluation-account for 41.6-48.3% of Full-AV unsafe outputs for which a risk source could be assigned. In the Cross-Modal Harm Emergence category, AV-Joint accounts for 87.5% of unsafe outputs withan assigned risk source. Together, these findings demonstrate the value of AV-SafetyBench for evaluating T2AV safety across the visual and audio modalities and their interaction.