Papers for

video post-production teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Video deraining improves by adapting restoration to uncertainty

Selective Generative Restoration for Video Deraining via Uncertainty-Aware Diffusion

Abstract: Video deraining is challenging due to rain streaks, occlusions, and atmospheric effects. Existing methods mainly follow a deterministic restoration paradigm, assuming that clean scenes can be uniquely recovered from rainy observations. However, rain degradation is spatially heterogeneous. Reliable regions require faithful reconstruction, whereas severely occluded regions contain ambiguous content that require stronger prior-driven inference. Therefore, conventional deterministic restoration often produces over-smoothed results, while global generative restoration may introduce unnecessary hallucinations. We introduce a Selective Generative Restoration paradigm for video deraining, where generative priors are adaptively activated according to restoration uncertainty. Based on this principle, we propose Selective Generative Video Deraining (SGVD), an uncertainty-aware diffusion framework that enables adaptive fidelity preservation and generative recovery. Specifically, SGVD exploits stochastic diversity between dual one-step diffusion predictions to estimate region-level restoration uncertainty, which determines where observation fidelity should be preserved and where generative inference should be activated. Furthermore, an anchor-guided trajectory consistency mechanism is proposed to regularize stochastic diffusion evolution across frames, reducing temporal generation drift and improving video coherence. To achieve practical efficiency, SGVD adopts one-step diffusion inference while maintaining uncertainty-aware generative capability. Extensive experiments demonstrate that SGVD achieves superior restoration performance, perceptual quality, and temporal consistency compared with state-of-the-art video deraining methods.

Thu 12 NovComputer Vision and Pattern Recognition
The gist
Cleaning rain from videos is tricky because some parts need careful restoration while others are too blurry or covered to fix directly. The authors developed a method that guesses how confident it can be about each part of the video and uses a special process to either keep what it sees or recreate missing details where needed. This approach prevents unnatural smoothness or random guessing and keeps the video looking natural and consistent across frames. Their system also runs efficiently, making it practical for real use.
Open → 2611.16310v1

Model removes burned in video text while preserving background details

Hint, Not Hole: Localization-Robust Video Overlay Removal

Abstract: Video overlay removal recovers the scene beneath burned-in subtitles, karaoke lyrics, titles, credits, logos and watermarks. Generative video models now fill a known region convincingly, but in practice the region comes from a detector or a segmenter, and it is wrong in both directions: it misses strokes, halos, whole frames and whole objects, and it covers background between characters, around boxes and on genuine scene text. Erasers that treat the mask as a hole, regenerating inside and copying outside, inherit every one of these errors. We present HintErase, which treats localization as a hint, not a hole. The model sees the full, unmasked video together with a hint channel; during training the hint is corrupted the way real localizers fail, while supervision follows the true overlay. The result is a single model that removes overlay the hint missed, keeps background the hint wrongly covered, and is trained to keep genuine scene text even when the hint lands on it. For full-length releases, a streaming pipeline handles interlaced sources, shot cuts, missed and jittering hints, paste-back and audio, with memory independent of the length of the video. This report describes the approach, the pipeline that turns it into full-length deliveries, and qualitative results; video results, with input, hint and output side by side, are on the project page.

Tue 10 NovComputer Vision and Pattern Recognition
The gist
Sometimes videos have text like subtitles or logos stuck onto their images, which can be tricky to remove perfectly. The authors created a method where instead of treating the detected text area as a hole to fill, their model sees the detection as just a hint and looks at the whole video to decide what to remove or keep. This way, their model removes text that the detector might have missed, avoids removing parts of the background it shouldn’t, and keeps any actual scene text intact. They also built a streaming system to handle long videos with complicated editing features smoothly.
Open → 2611.13711v1

Audio guided method improves facial video restoration and lip sync

TaoMate-FVSR:Audio-Guided Facial Video Restoration

Abstract: AI-generated videos often depict people within detailed scenes while leaving their small facial regions poorly resolved. Visual super-resolution can sharpen these regions without resolving whether the recovered lips and teeth agree with the spoken audio. We present an audio-guided facial video restoration framework that addresses this ambiguity within a reference-guided video flow model. Source--noise mixing initializes reconstruction from a partially preserved observation, and structured reference injection separates appearance anchors from continuous motion context. Training combines restoration of degraded images and videos with correction of degraded dubbed sources under target audio. Appearance and synchronization supervision allow the model to recover facial detail and support visual dubbing when the audio is replaced. On talking-head restoration benchmarks, our method achieves the best paired perceptual similarity and synchronization scores among the compared restoration methods. Component comparisons examine audio conditioning, the reconstruction formulation, and reference injection. Native AIGC examples further illustrate the recovery of lip contours, dental structure, and surrounding facial texture in scene-level footage. injection. Native AIGC examples further illustrate the recovery of lip contours, dental structure, and surrounding facial texture in scene-level footage.

Mon 2 NovComputer Vision and Pattern Recognition
The gist
Videos created by AI often show detailed scenes but have blurry faces, especially around the mouth. The authors created a new method that uses the voice audio to help fix face videos so the lips and teeth match the spoken words. Their system can restore small facial details and keeps lip movements synchronized even if the audio changes. Tests show it performs better than previous methods on talking-head videos. This approach helps make AI videos look more realistic and believable around the face.
Open → 2611.03393v1

Video restoration improves by preserving temporal noise correlations

Taming Noise Initialization for Zero-shot Video Restoration with Image Diffusion Priors

Abstract: Recent advances in diffusion models have achieved remarkable success in restoring high-quality images in a zero-shot manner. Unfortunately, extending this success to video restoration is challenging due to the substantial computational resources required to pre-train video diffusion models. A rising trend seeks to address video restoration by utilizing a pre-trained image diffusion model, initiating restoration from either frame-wise independent or identical Gaussian noise. Yet, these approaches often fall short, as such simple noise initiations fail to capture and utilize the inherent temporal correlations within degraded video frames. In this work, we propose taming noise initialization for zero-shot video restoration, which preserves the intrinsic temporal correlations by generating initial noises directly using DDIM inversion on the corrupted video frames themselves. Building on this, we introduce an adaptive noise blending strategy that suppresses degradation patterns encoded during DDIM inversion while gradually steering the noise toward a Gaussian distribution. To further emphasize temporal consistency, we propose measurement-guided trajectory attention to better align features across frames. Extensive experiments on multiple video restoration tasks demonstrate that our method outperforms existing zero-shot approaches, consistently achieving superior results on the DAVIS, REDS4, and Set8 datasets.

Tue 27 OctComputer Vision and Pattern Recognition
The gist
Fixing damaged video frames is harder than fixing images because video frames are related to each other over time. The authors show that starting the fixing process with smarter noise patterns that keep these time links helps restore videos better. They use a method called DDIM inversion to create these noises based on the damaged video itself, then slowly adjust the noise to make restoration more effective. They also add a way to improve how the computer pays attention to details between frames for smoother results. Their approach works better on popular video test sets than earlier methods.
Open → 2610.37290v1

Single-exposure raw video improves highlight and shadow details

High Dynamic Range Video Reconstruction from Single-Exposure Raw Sequences

Abstract: Due to the limited dynamic range of conventional image sensors, captured low dynamic range (LDR) video often suffers from highlight clipping and shadow detail loss, making high-quality high dynamic range (HDR) reconstruction from single-exposure sequences highly challenging without alternating exposures or extra hardware. Alternating-exposure HDR methods sacrifice frame rate and struggle with motion alignment, making them impractical for real-world capture. To address this, we propose RawHDRV, an end-to-end framework for single-exposure Raw video HDR reconstruction, that fundamentally exploits the linear response and channel-specific characteristics of Bayer data. Specifically, it features a channel-decomposition temporal alignment and fusion strategy that processes Bayer channels separately to exploit their distinct exposure characteristics, together with exposure-aware weighted fusion. It further incorporates an exposure complementarity mask-guided restoration module that leverages inter-frame exposure redundancy to adaptively fuse reliable information and suppress saturation artifacts, and introduces a mask-guided color loss that combines normalized error constraints with gradient smoothing to enhance highlight recovery. Furthermore, we construct a large-scale mobile Raw-HDR video dataset with per-frame HDR annotations. Experiments show that our method achieves the state-of-the-art results in all metrics, demonstrating superior spatial quality and temporal stability under extreme exposure conditions. The code is available at https://github.com/supeixian/RawHDRV.

Wed 23 SeptComputer Vision and Pattern Recognition
The gist
Cameras usually capture videos with limited brightness range, causing bright spots to be too bright and dark areas to lose detail. The authors introduce RawHDRV, a method that uses the raw sensor data from a single exposure to recover these details better without needing multiple exposures or special hardware. Their approach processes different color channels separately to handle brightness better and uses smart ways to combine multiple frames to improve the image quality. They also created a large dataset to test their method, proving it works well even in challenging bright or dark scenes.
Open → 2609.27274v1

Bidirectional flow improves 3D scene reconstruction and video quality

Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement

Abstract: Sparse-view 3D scene reconstruction with 3D Gaussian Splatting (3DGS) is inherently underconstrained. Plausible renderings can also coexist with erroneous Gaussian geometry, as errors in positions or depths may be concealed by opacity, scale, and appearance; we term this failure mode Geometry Cheating. Existing regularization methods constrain geometry but remain limited to observed views, while video-diffusion-based methods complete unseen views yet mainly use them as RGB pseudo-supervision, underusing motion and temporal priors and lacking explicit geometry supervision. We present Bi-FlowGS, which uses optical flow to bridge generative view completion and Gaussian geometry regularization. Our plug-and-play Video-to-Geometry Flow Distillation (V2G) distills temporal correspondence priors from restored videos into Gaussian geometry to alleviate Geometry Cheating. Conversely, Geometry-to-Video Flow-Guided Restoration (G2V) uses the current 3DGS geometry to guide temporally consistent video restoration, providing more reliable generative supervision. Together, V2G and G2V form an implicit bidirectional co-refinement process, enabling restored videos and the optimized 3DGS scene to iteratively improve each other. Experiments demonstrate improved rendering quality and geometric consistency across wide-baseline and unbounded 360° benchmarks.

Tue 15 SeptComputer Vision and Pattern Recognition
The gist
Reconstructing detailed 3D scenes from a small number of views is hard because multiple 3D shapes can look similar when rendered. The authors show that errors in 3D geometry can hide behind how the scene looks in images. They create a method where improved video sequences help fix the 3D shapes, and the better shapes help improve the videos, working together through motion information. This back-and-forth process leads to clearer, more consistent 3D scenes and videos even from limited viewpoints.
Open → 2609.17039v1