Screen content videos enhanced by spatial temporal multi scale network
Spatial-Temporal Multi-scale Network for Screen Content Video Quality Enhancement
Computer Vision and Pattern Recognition
Summary
Screen content videos, such as those with lots of text and graphics, are hard to improve using traditional video enhancement methods because their quick changes and high-detail areas break usual patterns. To fix this, the authors designed a new network called STM-Net that processes these videos in smart ways to keep important details and handle sudden scene changes. Their method splits the video into parts to avoid mixing up the details and looks forwards and backwards in time to understand changes without needing to know exactly when scenes switch. Tests show that this approach improves the video quality better than previous methods.
What this means in practice
- •For video streaming service teams: Improve visual quality of screen content videos with complex text and graphics during streaming without manual scene change detection.
- •For video conferencing software developers: Enhance screen sharing video quality by preserving fine details and managing abrupt transitions in real time.
Authors
Ziyin Huang, Sik-Ho Tsang, Xinyuan Qin, Yui-Lam Chan, Xueling Zhou, Feiyu Chen
Abstract
Different from natural videos, Screen Content Videos (SCVs) are characterized by abrupt motion, scene switches, and high-frequency details such as text and graphics. Conventional video enhancement methods, which rely heavily on temporal continuity, often suffer from performance degradation when processing SCVs due to the disruption of temporal correlations. To address these challenges, we propose the Spatial-Temporal Multi-scale Network (STM-Net), a novel framework specifically tailored for compressed SCV enhancement. Our approach integrates three complementary components: a Prior-Guided Spatio-Temporal Dispatcher (PG-STD) that routes input into three parallel streams to avoid feature contamination, a Bidirectional Temporal Feature Extraction (BTFE) module that adaptively handles abrupt transitions without explicit detection, and a Cascaded Multi-scale Feature Distillation (CMFD) module that preserves critical high-frequency details. Experimental results demonstrate that STM-Net outperforms state-of-the-art methods in both objective metrics and subjective visual quality, providing a robust solution for screen content artifacts. Code is available at https://github.com/HUANGZiyin1/STM-Net.