Audio video editing tool lets users change scenes and sounds together
AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
SoundComputer Vision and Pattern Recognition
Summary
Editing video and audio clips together is hard because you want to change some parts without messing up others. The authors created AV-FlowEdit, a tool that lets people edit audio and video by simply describing changes in words, without needing to retrain or reverse the AI model. Their method carefully keeps what should stay the same while making sure the changed parts in sound and picture match well over time. This makes it easier to do editing that feels natural and keeps audio and video in sync.
What this means in practice
- •For video editors: Edit both audio and video simultaneously using text prompts to ensure synchronized and consistent changes without retraining models or manual masking.
- •For sound designers: Modify audio events in videos via text prompts while preserving visual events and overall sync, improving creative audio-visual workflows.
- •For advertising teams: Create synchronized audio-video edits for ads through prompt-driven tools that require no model training, speeding up campaign production.$Commercial implications: Enables development of prompt-based editing software that advertising agencies can sell to speed audio-visual content customization.
Authors
Shaohua Pan, Xiao Liu
Abstract
Joint audio-video diffusion models generate synchronized content, but editing existing clips requires target changes without disrupting non-target structure or cross-modal correspondence. We present AV-FlowEdit, a training-free and inversion-free framework for prompt-driven audio-video editing. Our key insight is that source--target flow differences provide both cross-modal editing directions and preservation cues. Under shared perturbations, they jointly update the video and audio branches, supporting visual-only, audio-only, and joint edits while retaining pretrained cross-modal interactions. To suppress drift accumulated during sampling, Cross-Modal Event-Aware Spectral Trajectory Preservation combines flow saliency, aligned audio activity, multiscale low-frequency displacement, and diffusion-stage scheduling. This adaptively preserves unsupported regions while allowing intended audio-visual events. AV-FlowEdit requires no model training, external masks, or optimization-based inversion. Experiments show strong prompt alignment and temporal consistency, together with competitive audio quality and cross-modal coherence.