Capable video editing unlocked by transferring image editing skills
VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling
Computer Vision and Pattern RecognitionArtificial IntelligenceMultimedia
Summary
Editing videos is much harder than editing images because it needs lots of examples showing changes frame by frame, which are rare and costly to make. The researchers developed VINCIE-NExT, a method that uses knowledge from image editing, where many examples exist, to help edit videos. It works by turning video editing into steps involving images, allowing it to learn from both image and video data together. By showing an image example of the desired edit, the method applies similar changes to each video frame accurately. This way, video editing can improve without needing huge video editing datasets or special retraining.
What this means in practice
- •For video content creators: Enable flexible and high-quality video edits by guiding edits with image examples, reducing reliance on costly video editing datasets.
- •For mobile app developers: Integrate advanced video editing features into apps using image-based editing demonstrations for efficient video frame changes.$Commercial implications: Allows creation of powerful video editing apps that work well with limited video data, improving user experience and app competitiveness.
Authors
Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng
Abstract
Building a capable video editor remains significantly harder than a video generator: editing requires (source, instruction, edited) triplets that are prohibitively expensive to annotate and difficult to synthesize at scale, whereas image editing has already reached maturity with millions of such pairs readily available. In this work, we introduce VINCIE-NExT, a unified framework that transfers editing capability from images to videos through in-context visual demonstrations, alleviating the need for large-scale paired video editing data. VINCIE-NExT decomposes video editing into a structured chain of composable sub-tasks (Video -> Image -> Image -> Video), routing editing intent through the image domain and enabling scalable joint training from heterogeneous image and video corpora under a unified diffusion objective. An image editing pair, synthesized by the model or supplied by the user, is prepended as an in-context visual demonstration that serves as a spatial appearance blueprint for every output frame. To ground appearance edits across the interleaved context, we introduce a novel position encoding that links image demonstrations and video frames in a shared spatial coordinate system, enabling pixel-faithful propagation of appearance changes to every output frame. Chain-of-Editing further provides principled test-time scaling: by executing the sub-task chain as progressive diffusion stages, editing quality can be improved by investing additional compute without retraining. Comprehensive experiments on OpenVE-Bench demonstrate the state-of-the-art performance across diverse editing categories, with ablations confirming the effectiveness of each component.