Video editing guided by image editing for precise local changes
EditDistill: Is It Possible to Guide Video Editing with Image Editing?
Computer Vision and Pattern Recognition
Summary
Editing videos so that changes look natural and consistent across all frames is difficult. The authors present EditDistill, a method that first edits a single video frame using an existing image editor, then learns the changes made and applies them to the whole video. This approach uses a special descriptor to capture the edit and a video diffusion model to generate the edited frames while keeping the motion and layout intact. Their experiments show that this method produces high-quality video edits with good stability and detail.
What this means in practice
- •For video production teams: Create precise and consistent local video edits guided by single-frame image edits for visual effects and post-production.
- •For mobile app developers: Integrate easy-to-use video editing tools that leverage image editing results to enhance user control over video modifications.
Authors
Guojun Lei, Hong Li, Hongbing Yang, Lixue Gong, Chi Wang, Zheng Dong
Abstract
Recent advances in image editing have enabled fine-grained and highly controllable visual modifications. However, achieving similar levels of controllability in video editing remains an open challenge, particularly for precise local edits, dynamic temporal effects, and consistent lighting across frames.In this paper, we propose EditDistill, a framework that distills image-level visual transformations from edits to guide video editing.Specifically, given a source video and an edit prompt, we first generate an edited version of the initial video frame using an off-the-shelf image editing model. We then distill the visual transformation of this original-edited image pair into a compact representation termed an edit descriptor.This descriptor is injected into a video diffusion model through our lightweight fusion module, which steers the denoising process toward the desired edit. To align the video trajectory with the demonstrated image edit while preserving motion and layout, we train with edit-descriptor alignment and DINO structural consistency losses, and derive an equivalent inference-time guidance formulation under flow matching.Extensive experiments demonstrate that our method achieves strong performance on standard benchmarks and produces high-quality video editing results across diverse scenarios. For more details and videos, please refer to the anonymous webpage: https://yu-shaonian.github.io/EditDistill-Web/.