Training free real time video editing keeps source details intact

AR-Edit: Training-Free Autoregressive Video Editing without Inversion

Computer Vision and Pattern Recognition

Summary

Editing videos as they play, without seeing future frames, is hard but useful for things like live streaming. The authors discovered that a special kind of video model keeps some information stable, which lets them reuse parts of the video data for editing without needing extra work to invert or understand the original video. This means they can edit videos quickly and accurately while keeping the original content's look and feel. Their method, called AR-Edit, works in real time and doesn’t require training on each new video.

What this means in practice

  • For video app developers: Build apps that edit live video streams with high quality and no delay using training-free methods.$Commercial implications: Enables creation of real-time video editing features that can be sold as part of video streaming or social media apps.
  • For video processing engineers: Implement efficient streaming video editing pipelines that maintain original content fidelity without costly model retraining or inversion.

Authors

Hovhannes Margaryan, Quentin Bammey, Christian Sandor, Vicky Kalogeiton

Abstract

Editing a video stream in real time, without access to future frames, is essential for interactive applications. Existing training-free video editing methods, however, assume full offline access to the entire sequence, while streaming approaches either rely on additional training or are limited to specific editing techniques. We show that this limitation can be removed by analyzing the self-attention dynamics of DMD-distilled autoregressive (AR) video diffusion models. We uncover a key asymmetry: queries and keys remain stable and aligned with their source counterparts across denoising timesteps, while values diverge and encode the evolving content. Inherited from the non-distilled bidirectional teacher, this asymmetry is strongest at high noise, where few-step streaming models operate. The resulting mechanism is simple: a single forward pass suffices to capture source identity in the value features, which can be reused throughout generation. Starting from pure Gaussian noise and injecting only these source values reconstructs the input with fidelity matching or surpassing inversion-based methods, effectively eliminating inversion. Based on this insight, we introduce AR-Edit, a training-free, inversion-free method that caches source values once and selectively injects them during denoising. Across multiple AR backbones, AR-Edit enables real-time streaming editing with superior source preservation and tonal fidelity at minimal overhead.