Video overlay removal improves scenes obscured by text and logos
DeOverlay: A Benchmark and Source-Anchored Flow for Video Overlay Removal
Computer Vision and Pattern Recognition
Summary
Video overlays like subtitles, titles, and logos can cover up important parts of a video scene. This paper introduces a new method called DeOverlay that better removes these overlays by using a special flow model which carefully reconstructs the hidden scene. The authors also created a large benchmark dataset to test how well different methods work across many types of text overlays. Their results show DeOverlay removes overlays more accurately and keeps the videos looking natural compared to other methods. They will share their data and code for others to use.
What this means in practice
- •For video editing teams: Remove unwanted subtitles, titles, or logos from videos to restore clean scenes during post-production.
- •For content moderation teams: Detect and clean video content by removing distracting or unauthorized text overlays and graphics.
Authors
Deshui Miao, Yuhang Ma, Shaoyong Jia, Wenzhao Gao, Yue Zhang, Shaohui Jiao, Xin Li
Abstract
Video overlay removal aims to recover scenes obscured by subtitles, titles, and logos while preserving surrounding content. Existing methods use temporal propagation or source-conditioned generation, but can leave residual overlays and distort scene structure when useful source information is obscured. We propose DeOverlay, a source-anchored flow model that reconstructs the clean video from the source latent with fixed source guidance. Specifically, we train a shared velocity network along paired source-to-clean paths using region-weighted flow matching, enabling successive cor- rections from intermediate states. To select local source cues, we develop visibility-normalized temporal aggregation that excludes masked positions and averages the remaining observations. Global source context and edit-region embeddings complement these cues through additive condi- tioning, retaining a single video-token sequence. To evaluate scene recovery across diverse overlay conditions, we introduce DVTE-Bench, comprising 1,631 paired videos across eight text-overlay categories, with OCR-screened clean targets, frame-aligned pixel masks, and a common evaluation protocol. Experiments on the complete release show that DeOverlay outperforms four baselines on all five reconstruction and perceptual metrics, reducing Mask LPIPS and Mask VFID by 58.3% and 70.3%, respectively, relative to the strongest evaluated baseline. Additional qualitative examples demonstrate the removal of graphic logos and stickers. We will make the complete benchmark, evaluation code, and test outputs availabl. Project page:https://yahooo-m.github.io/DVTE-Bench/