Papers for

video editors

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Video tone mapping improved by learning to infer hidden brightness levels

Reframing Video Inverse Tone Mapping as Blind Posterior Inference via Latent Luminance Supervision

Abstract: High-dynamic-range (HDR) displays are now common, but most video content, such as archival footage, broadcast material, and the output of generative video models, exists only in standard dynamic range (SDR). The need is sharpest for generated video, which has no HDR master at all and is graded alongside live-action and visual-effects material finished in HDR. Converting SDR to HDR means undoing a tone-mapping operation that was never recorded, and clipping destroys the radiance wherever it acts, so HDR recovery is a blind inverse problem. Methods built on generative video priors fill clipped regions with plausible structure, but their reconstructions are systematically biased: in every public method we test, saturated content leans too bright or too dark, and the direction of the lean moves with the tone operator behind the input. We call this failure the luminance error, measure both its magnitude and its sign, and make it the focus of this work. Reframing ITM as posterior inference under an unknown operator locates the failure in two places and yields two corrections. A fixed affine functional of the video-diffusion latent reads log-luminance without decoding; on that coordinate an asymmetric loss selects an expectile of the posterior, applied only where clipping leaves luminance unconstrained. We also synthesize training SDR through an explicit family of tone operators, so the model marginalizes over what the observation cannot show. Trained this way, our luminance error stays flatter across operators and smaller on average than any public method, and the full system achieves the best perceptual quality on all three benchmarks.

Wed 11 NovComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Many videos are made for screens with less brightness range, but new displays can show much brighter colors and details. Changing a regular video to look good on a bright screen is tricky because we don’t know exactly how the video’s brightness was originally changed and some details are lost. The authors created a smart way to guess the original brightness using patterns inside the video and training the system on many examples with different brightness changes. This approach makes the bright parts of videos look more natural and accurate across different methods than before.
Open → 2611.15146v1

Clips2Story creates short videos from shuffled clips with keyword input

Clips2Story: Training-Free Video Storyboarding and Editing Using Multimodal Retrieval-Embedded Generation

Abstract: Video editors often work with large collections of fragmented footage and must select and organize relevant clips into a coherent story. However, many existing video editing and summarization methods assume a temporally ordered source video and aim to preserve its original narrative and temporal structure. In this paper, we introduce Clips2Story, a training-free, LLM-based pipeline that enables users to create keyword-specified short videos from a pool of video clips. Given a pool of clips and a user-specified keyword, we first extract text-based multimodal descriptors for each clip and retrieve the clips most relevant to the keyword. We then provide the textual descriptors of these candidate clips to an LLM, which selects and arranges them into an editable storyboard with connective narrations. Finally, we assemble the selected clips and retrieve accompanying visuals for the narrations to compose the output video. To evaluate the proposed pipeline, we construct a synthetic evaluation dataset from 100 long-form videos across five genres by segmenting each video into short clips using a scene-change detection model and shuffling the resulting clips to remove their temporal order. Through objective evaluation and qualitative examples, we demonstrate that Clips2Story can effectively retrieve keyword-relevant clips and compose coherent storyboards based on user-specified keywords. Human evaluation further shows that Clips2Story outperforms a strong baseline in terms of narrative coherence, keyword alignment, and overall realism.

Wed 4 NovComputer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
The gist
When video editors have many small clips out of order, it can be hard to make a good story from them. This paper presents Clips2Story, a tool that does not need training and uses a large language model (LLM) to pick clips related to a keyword and put them in a logical storyboard with narration. The tool then combines the clips and matching images to produce a short, understandable video. The authors show that Clips2Story works well through tests and human judgments, making it easier to create videos without needing the original sequence of footage.
Open → 2611.05992v1

VidDub improves video dubbing by syncing lip movement to new speech

VidDub: Replacement-Aware Distillation for Two-Step Video Dubbing

Abstract: Video dubbing aims to synthesize lip movements that match target speech while preserving the speaker identity, head pose, and background of a source video. Existing editors are predominantly trained via self-reconstruction under original paired audio, even though the source mouth becomes a conflicting target when driving speech is replaced. We introduce replacement-aware cross-audio supervision: paired-audio samples use full-frame reconstruction, whereas unpaired replacement-audio samples mask out source-mouth reconstruction losses and supervise the mouth solely through audio-visual synchronization while preserving unmasked facial regions and background. Because offline dubbing provides the complete source video and target speech beforehand, we present VidDub, a full-context diffusion editor distilled into two denoising steps (2 NFE). Our replacement-aware teacher first learns accurate lip synchronization without regressing toward conflicting source mouth movements. Matched trajectory regression then transfers the teacher's CFG-guided trajectories to a two-step student, and distribution matching distillation recovers visual quality after step compression. We will release the curated training dataset, VidDub-Bench, fixed paired- and cross-audio evaluation manifests, preprocessing outputs, and unified evaluation code. Across three training seeds, masking out conflicting source-mouth reconstruction improves cross-audio Sync-C by 0.833; the final two-step model combines strong lip synchronization with superior visual quality, identity preservation, and visual speech recognition across four benchmarks.

Mon 2 NovComputer Vision and Pattern Recognition
The gist
Video dubbing changes the lip movements in a video to match new spoken words while keeping the person’s identity and background the same. This is hard because the original mouth shapes conflict with the new speech. The authors introduced a way to ignore the old mouth shapes during training and instead focus on matching the lips to the new audio. They built a system called VidDub that does this efficiently in just two steps and tested it on several benchmarks, showing better lip syncing and overall video quality.
Open → 2611.03404v1

Motion transfer system anchors and routes motions to correct objects

Regional Anchor Routing for Multi-Object Motion Transfer

Abstract: Compositional motion transfer requires a model to extract motions from multiple reference videos, and apply each motion to a user-specified object in one target scene. Existing methods normally struggle to reliably assign different reference motions to the correct target objects, thus a motion intended for one object may affect another. Meanwhile, some motion-specific tuning approaches are still facing another issue: they must run a new optimization whenever the reference video changes, making repeated transfers inefficient. To address these limitations, we propose RART, a framework that unifies object-wise motion extraction, target-region anchoring, and frame-aligned injection into a frozen video Diffusion Transformer for multi-object motion transfer. For each reference motion, RART extracts temporally ordered features from the masked reference video, and then conditions them on the target first-frame mask to form mask-anchored motion tokens. Frame-aligned motion cross-attention allows each generated frame to read only the motion tokens at the corresponding time slots, composing all object motions in one denoising process. During training, a motion-attention routing loss encourages each target region to attend to its corresponding motion tokens and reduces interference between objects. Therefore, RART can handle novel reference-target combinations without per-video optimization. Experiments show that RART achieves accurate motion preservation, efficient inference, and strong compositional generalization.

Thu 29 OctComputer Vision and Pattern Recognition
The gist
Moving multiple objects in a video by copying their motions from different videos is tricky because motions can get mixed up between objects or need slow tweaking for each new video. The authors developed RART, a system that keeps motions tied to the right objects using special tokens anchored to object regions in the video. This system can combine different motions into one video without extra tuning for each new clip, making the process faster and more accurate.
Open → 2610.39478v1

MemoryFusion improves video fusion with long term memory and smoothing

MemoryFusion: Cross-Temporal Memory Learning for Multimodal Video Fusion

Abstract: Multimodal videos contain rich temporal cues that can facilitate faithful fusion. However, existing video fusion methods mainly rely on short-range temporal interactions, leaving valuable long-term information underexplored. Moreover, inconsistent fusion representations can cause flickering artifacts, undermining temporal coherence. To address these coupled challenges, we propose MemoryFusion, a two-stage framework integrating cross-temporal memory learning with progressive temporal refinement. Considering the complementary roles of adjacent and distant frames, the first stage combines a Short-Term Memory Bank for local temporal guidance with a Long-Term Memory Bank that preserves reliable historical information through a fusion-oriented update strategy. To mitigate temporal fluctuations, a Temporal Smoothing Module leverages soft motion cues for coarse background stabilization. Building upon the resulting fusion prior, The second stage further employs a lightweight 3D residual Temporal Refinement Module to correct remaining temporal inconsistencies while enhancing structural details. Extensive experiments demonstrate that MemoryFusion achieves superior fusion quality and temporal consistency over existing frame-based and video-based fusion methods. The source code is publicly available at https://github.com/Meiqi-Gong/MemoryFusion.

Thu 29 OctComputer Vision and Pattern Recognition
The gist
Multimodal videos combine information from different sources but existing methods focus only on short-term frame interactions, missing important long-term details. The authors propose MemoryFusion, a two-part system that remembers both recent and distant frames to create better combined video content. Their method also smooths videos over time to reduce flickering and improve stability. Tests show MemoryFusion produces clearer images and more consistent videos than prior methods.
Open → 2610.39322v1

Video and clicks improve sound separation for similar on-screen sources

SelectSep: Target-Selective Video Queries for Audio Source Separation

Abstract: Video-queried sound separation (VQSS) aims to isolate a target sound from an audio mixture using its corresponding video as a query. However, scene-level video queries can become ambiguous when multiple sounding objects are visible, particularly when the target and interferer are semantically similar. We propose SelectSep, a target-selective VQSS framework that jointly uses text and a single click coordinate to specify the target source and instance without requiring segmentation at inference time. We adapt the visual encoder through target-feature distillation with text and click conditioning, and then fine-tune the separator using the adapted visual features. We also introduce an on-screen interference setting, where both the target and interferer are visible, to evaluate target selectivity. Experiments on standard benchmarks and the on-screen interference setting demonstrate consistent improvements in separation quality and target selectivity, particularly for mixtures containing semantically similar sources.

Wed 28 OctMultimediaSound
The gist
Separating sounds from a video with multiple people or objects making noise is tricky when they look alike. The authors propose SelectSep, which uses a short text description plus a single click on the video to pick out the sound of just one target object. This approach helps the computer focus better on the intended source without needing detailed segmentation of the image. They tested it on examples where both the target and interfering sounds are visible and similar, showing improved sound separation results.
Open → 2610.38624v1

Video matting model improves smooth edges across long video clips

VideoMaMa++: Temporally Consistent Video Matting via Preserve-and-Refine Token

Abstract: Diffusion-based video matting has recently emerged as a leading approach, leveraging video diffusion priors from internet-scale pretraining to achieve strong zero-shot generalization from limited synthetic data. However, the heavy compute of these diffusion backbones limits inference to a short chunk of just 6 to 24 frames, so even ordinary videos span multiple chunks. Because each chunk is matted independently from coarse mask guidance, existing methods fail to maintain consistency across chunk boundaries, producing flickering edges and discontinuous transparency. We address this with a mask-guided diffusion-matting framework built on two ideas. The RGB and mask conditioning are represented as separate token streams that interact through full spatio-temporal self-attention, letting the model adaptively combine fine RGB texture with coarse mask layout rather than committing to a fixed channel-wise fusion. On top of this, we propose a novel pair of learnable embeddings, the preserve token and the refine token, that act as a per-frame conditioning interface and enable a sliding-window inference scheme in which already-generated mattes are propagated across chunk boundaries for local temporal consistency, complemented by a globally shared reference anchor at every chunk to prevent error accumulation in long sequences without any additional training. Across standard video matting benchmarks and in-the-wild long footage, our framework outperforms prior diffusion-based methods in both fine-detail accuracy and temporal consistency, and uniquely maintains consistency across chunk boundaries on long videos.

Mon 19 OctComputer Vision and Pattern Recognition
The gist
Video matting is a process that separates a foreground subject from the background in videos. Existing methods that use diffusion models perform well but only on short video segments, which causes flickering and mismatched edges when the video is longer. The authors propose a new approach that uses special tokens to carry information between video chunks, maintaining consistent and smooth edges across the whole video. Their method also combines detailed color textures with rough masks more flexibly, improving the quality and temporal consistency of the matting results.
Open → 2610.26857v1

Video subtitle removal benchmark improves fine detail and consistency tests

MFVSR: A Multi-Style and Fine-Grained Benchmark for Video Subtitle Removal

Abstract: Video subtitle removal (VSR) is an important video restoration task, yet existing evaluation remains limited by low-resolution datasets, fragmented test sets, and generic video-inpainting protocols that fail to adequately capture localized restoration errors, unintended modifications, temporal instability, and incomplete outputs. In this paper, we introduce MFVSR, a large-scale multi-style benchmark for fine-grained and reliability-aware video subtitle removal evaluation. MFVSR distinguishes itself through three key features: (1) Large-scale and diverse data, comprising 3,000 paired videos from Real-HQ, Real-LQ, and AI-Generated sources, with an auxiliary set of 953 in-the-wild real videos for real-world coverage analysis; (2) Controlled subtitle diversity, spanning 16 languages and four factors covering spatial layout, temporal behavior, visual style, and language mode; (3) Fine-grained evaluation, introducing an information-conditioned, spatially decoupled protocol that separates blind from mask-guided removal and measures Restoration Fidelity, Preservation Fidelity, Temporal Fidelity, coverage, and failures. Experiments on six representative methods reveal substantial discrepancies between local restoration, outside-region preservation, temporal consistency, and full-set reliability that are obscured by conventional full-frame metrics. MFVSR therefore provides a more comprehensive and diagnostic foundation for evaluating and advancing video subtitle removal methods.

Tue 13 OctComputer Vision and Pattern Recognition
The gist
Removing subtitles from videos is tricky because existing tests use low-quality videos and simple checks that miss errors like parts left behind or flickering between frames. The authors created MFVSR, a big and diverse video set with many subtitle types and languages, plus new ways to measure how well subtitles are removed without damaging the video. They tested six different subtitle removal methods and found some common issues that old tests overlooked. This benchmark can help make subtitle removal tools better and more reliable.
Open → 2610.18461v1

QK Edit improves multimodal transformers for image and video editing

QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

Abstract: Multimodal Diffusion Transformers (MM-DiTs) have recently emerged as a powerful framework for unified text-vision synthesis, surpassing traditional U-Net architectures in generative tasks. One key innovation lies in its Multimodal Self-Attention (MM-SA) interaction, where image and text tokens are concatenated and processed via self-attention. However, this mechanism poses significant challenges for editing, rendering conventional U-Net-based attention manipulation methods ineffective. To address this limitation, we propose QK-Edit, a training-free framework that exploits the unique attention dynamics of MM-DiTs for precise text-guided image and video editing. By introducing a novel query-key manipulation strategy, our method isolates and adjusts critical attention components to achieve an optimal balance between prompt fidelity and structural consistency. This enables seamless edits across various tasks, including object addition, object removal, object replacement, changing background, changing material, changing color, and style transformation. Notably, it can be easily implemented with feature replacement in inference. QK-Edit demonstrates superior editing performance on state-of-the-art models, such as FLUX and HunyuanVideo, effectively bridging the gap between generative power and editable flexibility in MM-DiTs, and paving the way for scalable multimodal content creation.

Tue 13 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Editing images and videos guided by text has gotten better with new transformer-based models that combine text and vision. But these models have made traditional editing tricks less effective. The authors introduce QK-Edit, a method that tweaks how attention works inside the model to make precise edits based on text without retraining. This helps add or remove objects, change colors, and alter styles smoothly in both images and videos.
Open → 2610.17829v1

Audio video editing tool lets users change scenes and sounds together

AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation

Abstract: Joint audio-video diffusion models generate synchronized content, but editing existing clips requires target changes without disrupting non-target structure or cross-modal correspondence. We present AV-FlowEdit, a training-free and inversion-free framework for prompt-driven audio-video editing. Our key insight is that source--target flow differences provide both cross-modal editing directions and preservation cues. Under shared perturbations, they jointly update the video and audio branches, supporting visual-only, audio-only, and joint edits while retaining pretrained cross-modal interactions. To suppress drift accumulated during sampling, Cross-Modal Event-Aware Spectral Trajectory Preservation combines flow saliency, aligned audio activity, multiscale low-frequency displacement, and diffusion-stage scheduling. This adaptively preserves unsupported regions while allowing intended audio-visual events. AV-FlowEdit requires no model training, external masks, or optimization-based inversion. Experiments show strong prompt alignment and temporal consistency, together with competitive audio quality and cross-modal coherence.

Fri 9 OctSoundComputer Vision and Pattern Recognition
The gist
Editing video and audio clips together is hard because you want to change some parts without messing up others. The authors created AV-FlowEdit, a tool that lets people edit audio and video by simply describing changes in words, without needing to retrain or reverse the AI model. Their method carefully keeps what should stay the same while making sure the changed parts in sound and picture match well over time. This makes it easier to do editing that feels natural and keeps audio and video in sync.
Open → 2610.13072v1

Hybrid cinematography helps filmmakers safely redo camera moves

Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting

Abstract: On a film set, the camera move is committed during a take. Generative video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept hallucination. We demonstrate the workflow through a mobile augmented reality application for on-set planning, capture, and review, and an offline pipeline for existing video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination.

Thu 8 OctHuman-Computer InteractionComputer Vision and Pattern Recognition
The gist
Filmmakers usually have to decide exactly how to move a camera during shooting and can't change it later. This paper introduces a method that combines real shooting with computer-generated video so filmmakers can change camera moves after filming. It shows where the computer might have to guess or 'hallucinate' parts of the video not originally recorded, so filmmakers can plan better or accept some guesswork. They built an app to preview and manage these risks in real time, helping filmmakers make smarter choices on set.
Open → 2610.12455v1

CrossEdit enables flexible editing across images audio and video

CrossEdit: Cross-Modal Training Enables Rich Audio-Visual Editing

Abstract: Multimedia editing requires generative models to interpret complex, compositional instructions and modify only the desired elements across modalities, a capability largely beyond existing systems. Current approaches are typically limited to simple, single-attribute edits within one modality, since diverse, high-quality editing pairs are hard to obtain, especially for cross-modal tasks where aligned audiovisual (AV) data is scarce. We present CrossEdit, a unified omni-modal editing model for images, audio, and video that performs AV movie scene edits zero-shot through cross-modal transfer. Our key observation is that complex instruction-following, once learned in any modality, generalizes to others. We exploit the additive nature of acoustic signals to procedurally generate large-scale audio editing pairs with compositional instructions, and show that fine-tuning on this synthetic data and self-supervised AV masked reconstruction, alongside a targeted set of curated cross-modal tasks, induces instruction-following that transfers zero-shot to unseen modality and instruction combinations. To evaluate this capability, we release CrossEditBench, a human-annotated benchmark of AV edits on movie scenes, and propose AV-FES, a metric that jointly scores instruction following and consistency. We show that our proposed techniques improve performance on zero-shot AV movie scene editing, while maintaining or improving performance on lip-synced speech editing and standard image, video, and audio editing benchmarks. Demos are available at https://wanchichen.github.io/crossedit/.

Wed 7 OctSound
The gist
Editing multimedia content like pictures, sound, and video often requires changing only certain parts based on detailed instructions. Existing tools usually handle simple changes in just one type of content and struggle when sound and visuals must be edited together. The authors created CrossEdit, a single model trained to follow complex instructions and make edits across images, audio, and video without needing direct examples in every case. They used clever techniques to generate training data and showed that the model can apply what it learned from one type of media to others, improving editing quality on movies and other benchmarks.
Open → 2610.10264v1

Memory guided system creates b roll sequences from user videos

Memory-Guided B-Roll Generation from User Video Collections

Abstract: We introduce an approach for collection-grounded B-roll sequence generation. Given a user's video collection, a directive given in natural language, and a target duration, the goal is to produce a multi-shot sequence that complements the user's primary footage (A-roll) while preserving the collection's characters, settings, objects, and style. This task is challenging as one must choose the visual evidence from hours of captured footage that should guide the generation of each shot in the sequence. We address this challenge with MemComposer, a three-stage system that turns raw footage into a structured memory with visual references (characters, settings, objects, and style) and uses it to plan, retrieve, and generate grounded B-roll sequences. First, in a one-time offline stage, MemComposer constructs an entity-centric memory from raw video. Second, it uses the memory and user directive to plan a grounded sequence and retrieve conditioning frames for each shot. Third, it iteratively generates and critiques the sequence to enforce identity, setting, and sequence-level consistency. We evaluate MemComposer in a user preference study along two dimensions: prompt adherence and visual alignment to the user's collection. Against an ungrounded text-to-video planner, MemComposer wins 60.0\% of prompt-adherence and 92.8\% of visual-alignment comparisons, showing the grounding benefit of collection memory and reference retrieval. Against retrieval-only sequences assembled from captured footage, MemComposer wins 94.5\% of prompt-adherence comparisons, showing the value of generating missing shots, while retrieval-only sequences are preferred for visual alignment in 58.2\% of comparisons.

Thu 1 OctComputer Vision and Pattern Recognition
The gist
Editing videos with interesting extra footage is hard when you have hours of clips to choose from. The authors introduce MemComposer, a tool that learns what’s in a user’s video collection and then plans and generates new video sequences that match the people, places, and style seen before. It picks the best frames and makes sure the new sequences fit what the user wants and look consistent. Their tests show MemComposer produces better matching and more relevant sequences than other methods that don’t use this memory or only reuse existing clips.
Open → 2610.01884v1

Video to speech system uses text to improve accuracy and naturalness

Watch Your Speech: Text-aware Video-to-Speech Synthesis with Textual Conditioning

Abstract: Video-to-speech synthesis aims to generate natural-sounding speech from silent talking-face videos while ensuring phonetic accuracy. A fundamental challenge in this task is the inherent one-to-many mapping problem, where visual dynamics often lack sufficient information to uniquely determine the corresponding utterance. To address this, we propose Watch Your Speech (WYS), a video-to-speech synthesis framework that incorporates textual conditioning as an explicit linguistic cue to mitigate visual ambiguity. Our framework features an attention-based embedding fusion module that synergistically integrates textual context with video sequences, coupled with a conditional flow matching objective for high-fidelity speech generation. Extensive experiments on the LRS2 and LRS3 datasets demonstrate that WYS achieves superior performance, establishing new state-of-the-art results in audio-visual synchronization (LSE-C/D) while maintaining highly competitive textual accuracy (WER). Subjective evaluations further confirm that our model generates speech with near-human naturalness, validating the effectiveness of textual conditioning in content-controlled video-to-speech synthesis. Project page: https://github.com/gunwoo5034/Watch-your-Speech

Thu 1 OctComputer Vision and Pattern RecognitionSound
The gist
Turning silent videos of people talking into speech is tricky because the mouth movements alone don’t always show exactly what’s being said. The authors propose a new method that adds the actual text of what should be spoken as an extra clue. This helps the model figure out the right words and sounds more accurately and naturally. Their experiments found that this approach makes the generated speech match the video closely while sounding almost human.
Open → 2610.01012v1

Soundwich generates video with separate editable audio tracks

Soundwich: Video Generation with Layered and Controllable Audio

Abstract: Recent joint audio-video generative models can synthesize realistic videos with synchronized sound, but typically generate audio as a single mixed track. This limits source-level control and differs from practical audiovisual workflows, where speech, music, sound effects, and ambient sounds are represented as separate editable tracks. We introduce Soundwich, a training-free framework that transforms a frozen joint audio-video flow-matching model into a generator of multiple synchronized, independently editable audio stems coupled to a shared video. Soundwich generates separate audio stems with explicit control over their temporal activity. To keep separately generated sounds coherent, we introduce a shared scene representation that communicates global audiovisual context across stems while preserving their source-level separation. We further route cross-modal interactions between each audio stem and its corresponding visual source, improving audiovisual consistency. The resulting stems remain synchronized with the video and can be independently retimed, muted, replaced, or remixed. Experiments and human evaluations show improved temporal control, source separation, and naturalness, while enabling flexible source-level editing within coherent audiovisual generation. Code is available at https://github.com/CodyNing/Soundwich.

Wed 30 SeptComputer Vision and Pattern RecognitionMultimediaSound
The gist
Creating videos with matching sounds is common, but usually the sounds come as one mixed track, making it hard to edit parts like speech or music separately. The authors made Soundwich, a tool that splits sounds into separate tracks for speech, music, effects, and more, all perfectly timed with the video. It keeps these sound tracks connected so they make sense together, yet lets you change, mute, or move any part on its own. This approach helps people work with video and sound like in real studios, letting creative editing while keeping everything synced.
Open → 2610.00691v1

Selective video unlearning removes concepts while preserving the scene

FOMO: Forget the Concept, Don't Miss Out on the Scene in Selective Video Unlearning

Abstract: The rapid advancement of generative video models has enabled the synthesis of increasingly realistic and temporally coherent videos, while also raising concerns about the generation of harmful content. The reliance on large-scale web datasets during training inevitably exposes these models to undesirable material, making concept unlearning an essential mitigation. Existing methods mainly target static visual concepts, such as objects, identities, or unsafe appearance, largely overlooking motion unlearning. Furthermore, these approaches often pay little attention to preserving the surrounding scene. As a result, successful concept removal may unintentionally alter the background, composition, or overall video dynamics. We argue that effective unlearning should ideally change only what is targeted, while minimizing unnecessary changes to the remaining scene. In this work, we introduce FOMO, to the best of our knowledge the first training-based selective video unlearning method that directly treats preservation of the original scene as a priority. We formulate unlearning around two complementary objectives: what to change and what to preserve. Our method localizes concept-related representations and modifies them, while the preservation mechanism maintains non-target scene information without requiring auxiliary data. Beyond simply erasing unwanted concepts, FOMO explicitly redirects the generation toward a specified safe alternative. We further extend this formulation to motion unlearning, where the concept is defined by temporal behavior rather than a fixed spatial region. Our solution achieves effective unlearning across unsafe content, object, and motion concepts, while achieving the best trade-off between concept removal and scene preservation. Code: https://github.com/gmum/FOMO Project Page https://gmum.github.io/FOMO

Wed 30 SeptComputer Vision and Pattern Recognition
The gist
Videos created by AI can sometimes include unwanted objects, identities, or even unsafe content. The authors developed a method called FOMO that removes just the unwanted parts from videos without messing up the rest of the scene. Unlike previous methods, FOMO also works by removing certain movements or actions in videos. This means it can keep the background and overall look mostly unchanged while erasing only the target content.
Open → 2609.39605v1

Video object removal method improves background filling and smoothness

TripleFlow: Training-Free Video Object Removal by Bridging Residual Editing and Native Generation

Abstract: Video object removal presents a uniquely difficult editing challenge. Because a removal prompt specifies only what to erase rather than what to generate, the model must infer and reconstruct a highly specific occluded background entirely from the surrounding context. Existing training-free methods struggle with this because their editing mechanisms act primarily as localized erasers. They fail to actively synthesize the missing background details and often leave behind ghosting artifacts. To solve this, we propose TripleFlow, a training-free framework that tightly couples erasure and generation. It coordinates a source flow, a residual flow, and a synthesis flow throughout the entire process. By reusing a single target prediction, the residual flow isolates and suppresses the object, while the synthesis flow independently reconstructs the occluded background. Crucially, TripleFlow injects this newly synthesized background back into the editing trajectory at every step. This continuous feedback loop ensures that the generated structures actively guide the removal process, achieving seamless completion that is spatiotemporally consistent with the unedited scene. Extensive evaluations across five challenging benchmarks demonstrate that TripleFlow establishes a new state-of-the-art, significantly outperforming existing baselines in both reconstruction fidelity and temporal consistency.

Wed 30 SeptComputer Vision and Pattern Recognition
The gist
Removing objects from videos is hard because the system needs to guess and redraw the hidden parts behind those objects. Existing methods mainly erase the object locally but do not recreate the missing background well, causing leftover ghosts or blurs. The authors propose TripleFlow, which combines object removal and new background creation at every step, so the video looks smooth and consistent over time. This method works well without extra training and beats previous techniques on several tests.
Open → 2609.39157v1

Reusable personalized color editing method adapts photo look quickly

PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences

Abstract: Photographic color editing is inherently personal: the same image can appear too warm, too muted, or already satisfactory to different users. Most lookup table (LUT) and reference-guided methods target a specified appearance rather than model persistent preferences from repeated user choices. To address this gap, we introduce PrefLUT, a reusable and refinable user-preference modeling framework for deployable 3D LUTs, encoding ordered preferred/non-preferred image pairs into a lightweight Reusable User Profile that is reused across queries and refined using additional user preference pairs, without per-user optimization. A Query-Conditioned LUT Predictor combines this profile with each image to predict a LUT latent vector and edit strength. An Identity-Residual LUT Decoder and Edit-Strength Controller then produce an exportable 3D LUT. Experiments on three datasets demonstrate effective personalized editing and general-purpose enhancement. Each quantized profile requires only 260 bytes, and editing takes 1.365 ms/image on an RTX 5090 GPU. We also introduce the Preference-Conditioning Verification Protocol (PCVP), an evaluation protocol to verify whether personalized image edits depend on user preferences and the query image through controlled changes to user profiles, preference orders, pair correspondences, and query images.

Mon 28 SeptComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
People often like different color edits on the same photo, but usual editing tools target just one fixed look. The authors created PrefLUT, which learns a user’s color preferences from pairs of images they like or dislike, stores these in a tiny profile, and reuses or updates it easily to edit new photos. This approach works fast and efficiently on powerful GPUs and can create standard color lookup tables used in photo editing software. The authors also designed a test to make sure the edits truly reflect user preferences and the pictures involved.
Open → 2609.34133v1

CraftTrace enables flexible editing of full videos using video structures

CraftTrace: Unflattening Videos into Malleable, Creation-Inspired Structures for Generative Editing

Abstract: Recent generative video editing models enable video content modification (e.g., changing a character) but target short clips. Extending them to full multi-shot videos requires tedious work to locate relevant content across shots, segment it into clips, craft context-aware editing prompts for each clip, and repeatedly articulate complex editing intent. To address this, we explore an interaction paradigm for editing through underlying video structures (e.g., scripts, scenes, characters, shots, and their relationships). We present CraftTrace, an interactive prototype that transforms a video into a malleable, multilevel structure for generative editing. Users work in task-centric workspaces to modify elements or reshape relationships, while an AI agent translates and propagates changes across the video. A user study and expert review show that this structure helps users understand videos, formulate and refine editing intent, and explore alternatives, supporting rapid prototyping during early-stage exploration and full video post-production.

Thu 24 SeptHuman-Computer Interaction
The gist
Editing videos with many scenes and shots is hard because you must find and change parts one by one. The authors introduced CraftTrace, a tool that turns a whole video into a map of scenes, characters, and shots you can easily change. An AI helps apply your edits across the whole video so you don’t have to do repetitive work. This makes exploring ideas and refining edits faster and easier.
Open → 2609.30623v1

SSE model enhances and remixes audio using video and text guidance

Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing

Abstract: We introduce Spot, Separate, and Enhance (SSE), the first multimodal, user-guided generative model for audio remixing and enhancement. SSE enhances video content by rebalancing the audio, removing unwanted audio sources, and reducing reverberation, guided by both video and textual descriptions. To support its training and evaluation, we propose DegradedMix, a new dataset built on the audio remixing benchmark MuddyMix. We also adopt evaluation metrics from generative modeling, which better capture the creative nature of remixing than standard reconstruction-based metrics. SSE outperforms existing baselines in both controllability and remixing quality, as shown by extensive experiments. Project page: https://sse-ai.notion.site

Thu 24 SeptSoundArtificial Intelligence
The gist
Mixing audio in videos is hard when unwanted sounds or echo get in the way. The authors created a new system called SSE that uses the video and text descriptions to find, separate, and improve sounds in a video’s audio track. This lets people remove unwanted noise, balance sound levels, and reduce echoes with simple instructions. They also made a new dataset to help train and test their system and showed it works better than existing methods.
Open → 2609.29169v1

Lightweight model separates singing voices using audio and video cues

MambaVoice: Lightweight Audiovisual Singing Voice Separation Via A Hybrid Mamba-Transformer Model

Abstract: Isolating a target singing voice from a music video remains challenging, particularly in the presence of multiple vocalists and dense instrumental accompaniment. We propose MambaVoice, a lightweight audiovisual framework that leverages a hybrid Mamba--Transformer architecture for targeted singing voice separation. The model jointly encodes audio and visual streams using an attention-based band-split audio encoder and a spatio-temporal graph convolutional network (ST-GCN) for facial motion features. These modalities are fused through a multiplicative gating mechanism, enabling visual cues to selectively modulate audio representations. The fused features are processed by a hybrid backbone that combines Transformer self-attention with Selective State Space Models (SSMs), achieving efficient long-range temporal modeling with linear complexity. We evaluated MambaVoice on the Acappella and URSing datasets under challenging conditions, including mixtures with interfering singers. At 16.2 million parameters, the model demonstrates comparable performance, achieving 14.18 dB SDR on Acappella and strong cross-dataset performance on URSing, comparable to larger models at a fraction of the parameter count. These findings highlight the effectiveness of hybrid SSM--attention architectures for scalable, efficient audiovisual source separation, suggesting they are well-suited as lightweight components within larger pipelines. We conduct a perceptual study that further supports our improvements in objective metrics. We provide our implementation online.

Tue 22 SeptSound
The gist
Separating a singer’s voice from music videos is hard, especially when many people sing or lots of instruments play. The authors created MambaVoice, a compact model that uses both sound and face movements to pick out the target singer’s voice. It combines audio and video information in a clever way to focus on the right voice, and uses advanced methods to understand long stretches of music efficiently. Tests show MambaVoice works well compared to bigger models but with fewer resources.
Open → 2609.26635v1

Visual and text guided method removes sounds linked to video objects

TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculum

Abstract: Visual object removal can eliminate a target from video frames, yet its acoustic trace persists in the soundtrack, causing obvious audio-visual inconsistency. Existing video inpainting models operate solely on pixels, while audio editing models, especially for the sound removal task, are typically driven by text and therefore rely on limited single-modal control, which is less effective than multimodal guidance that provides stronger semantic grounding and temporal synchronization cues. In this paper, we present Text-Visual Guided Sound Removal (TV-AudioRemover), a target sound removal framework that leverages the visually edited video together with a natural-language instruction to suppress the sound associated with the removed visual object from the original audio mixture. To acquire high-quality training data, we devise a pipeline to construct a million-scale dataset of single-object audio-visual aligned samples, from which we synthesize mixture-target pairs customized for model training. To effectively leverage visual context and follow instruction intent, we augment the model architecture with task tokens, generalizable instruction modeling, and modality-specific global guidance. We further adopt multi-task training to strengthen task-role comprehension, and employ a hard-mixture curriculum that leverages semantically similar acoustic mixtures during fine-tuning to enhance fine-grained source discrimination. To support evaluation, we present AV-Remove-Bench, a comprehensive audio-visual object removal benchmark, along with dedicated objective metrics and an MLLM-based evaluation protocol. Experiments demonstrate that our method achieves state-of-the-art performance on both subjective and objective metrics. Project page: https://yjx-research.github.io/TV-AudioRemover/.

Tue 22 SeptMultimediaComputer Vision and Pattern RecognitionSound
The gist
When people remove objects from a video, the sounds those objects made often remain, making the video feel strange. The authors created a method that uses both the edited video and text instructions to remove the sounds linked to the removed objects. They trained their model on a large set of video and audio clips with aligned objects and sounds. Their approach improves how well the unwanted sounds are canceled while keeping the rest intact. They also created a new benchmark to measure how well these methods work.
Open → 2609.25864v1

Qwen3.8-Omni improves multimodal AI for long tasks and video work

Qwen3.8-Omni: Towards Native Omni-Modal Agents

Abstract: We introduce Qwen3.8-Omni-Flash, a natively multimodal agentic model for real-world multimodal productivity. Compared with previous omni models, which primarily emphasized perception and interaction, Qwen3.8-Omni-Flash substantially improves multimodal understanding and reasoning, as well as performance on long-horizon agentic tasks. These capabilities are supported by a native multimodal co-training strategy that preserves strong text-domain capabilities while facilitating the transfer of agentic capabilities from text to audio and video tasks. The model inherits the sparse mixture-of-experts (MoE) architecture of Qwen3.8-Next and extends the context window to one million tokens, supporting long-context multimodal reasoning and long-horizon planning. These advances enable integration into production workflows as a primary agent or a specialized sub-agent, supporting video editing, long-form audio and video translation, music-conditioned music video or movie generation, and video-based note or omni-skill creation. To address the lack of native audio and video support in existing agent harnesses, we release Qwen-MM-Plugins, a lightweight open-source plugin framework for multimodal productivity. We further frame real-time multimodal interaction as a system-level challenge requiring orchestration of context and memory management, tool use, and sub-agent delegation. Accordingly, we release Qwen-Live-Harness, an open-source framework for building responsive, real-time multimodal agents based on Qwen3.8-Omni-Flash. Extensive evaluations demonstrate that Qwen3.8-Omni-Flash achieves strong performance across multimodal understanding, reasoning, long-horizon agentic execution, and video productivity tasks. These results and the accompanying open-source tools support Qwen3.8-Omni-Flash as a practical foundation for deploying natively multimodal agents in research and production.

Tue 22 SeptComputation and LanguageComputer Vision and Pattern RecognitionMultimedia
The gist
Handling different types of information like text, audio, and video at once is hard for AI. The authors developed Qwen3.8-Omni-Flash, an AI model that can understand and reason about multimodal data over very long conversations or projects. It also helps with tasks like video editing and music video creation. They created new software tools to support real-time interaction and seamless work across different media types. This model and tools aim to make AI assistants better at complex, long-lasting tasks involving various media.
Open → 2609.25611v1

Edit VAR improves text guided video editing with better source and motion control

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

Abstract: Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages. Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.

Fri 18 SeptComputer Vision and Pattern Recognition
The gist
Editing videos by changing their content based on text is tricky because you need to keep parts of the video unchanged and consistent over time. The authors introduce Edit-VAR, a method that does this without needing extra training or reversing the video editing process. It smartly replaces parts of video data based on text, keeping unedited parts intact and making sure moving details stay smooth. Tests show it works better than existing methods in making edits that look good, keep the original video’s feel, and run efficiently.
Open → 2609.21268v1

Multi subject video editing improves with mask depth and noise control

MDN-Control: Mask-Depth-Noise Guided Region Control for Multi-Subject Video Editing

Abstract: Multi subject video editing modifies designated subjects while preserving non target content, but faces cross subject attribute leakage, and occlusion ambiguity. Existing approaches rely on masks and struggle to distinguish overlapping subjects or ensure consistent generation. To address these limitations, we propose MDN-Control, a training free framework jointly controlling target localization, occlusion geometry, and appearance initialization. Specifically, mask-guided localization provides consistent target localization, while depth-aware occlusion control resolves ambiguous boundaries between overlapping subjects. We further introduce noise latent prompting, which retrieves Gaussian initializations from a noise library for prompt relevant priors. Experiments on MSVBench show that MDN-Control achieves the lowest CM-Err and the highest Q-Edit, while maintaining competitive text alignment and temporal consistency, demonstrating the effectiveness of combining spatial, geometric, and latent priors for multi subject video editing.

Tue 15 SeptComputer Vision and Pattern Recognition
The gist
Editing videos with multiple people can be tricky when subjects overlap or cover each other. The paper's authors developed MDN-Control, a method that uses masks to find targets accurately, depth info to handle overlaps, and special noise patterns to start the editing. Their approach helps keep changes only on intended subjects while maintaining video consistency. Tests show their method works better than others on videos with multiple subjects.
Open → 2609.16475v1