Video super-resolution improves by tracking pixel motion over multiple frames
PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories
Computer Vision and Pattern Recognition
Summary
Making videos sharper and smoother between frames is tricky because the system usually guesses motion using just two key frames, leading to errors. The authors designed a method that looks at a longer sequence of video to better understand motion and represents each pixel’s movement as a single smooth path between frames. Using this approach, they can generate higher-quality detailed video frames at any moment in time. Their technique balances quality and computational efficiency for improving video resolution over space and time.
What this means in practice
- •For video streaming services: Enhance streaming video clarity by generating intermediate frames at higher resolution using long-term pixel motion tracking.$Commercial implications: Enables streaming providers to offer higher-quality video playback with smoother motion, improving user experience and increasing market competitiveness.
- •For video editing software developers: Implement video frame interpolation tools that maintain consistent pixel trajectories for better slow-motion and frame rate conversion effects.
Authors
Wanqi Yuan, Nianyi Li
Abstract
Existing continuous space-time video super-resolution methods often infer motion from only two endpoint frames and model the two motion directions independently, making them vulnerable to ambiguous correspondence and inconsistent trajectories. We introduce a continuous STVSR framework that combines long-context encoding, continuous HR feature lifting, and an endpoint-anchored PixelSpline. Long-range observations provide more reliable endpoint representations, while continuous upsampling constructs detailed appearance features directly on the requested HR grid. PixelSpline explicitly models each pixel's motion as a single continuous trajectory shared by both endpoints. Using this trajectory, our method warps endpoint colors into a coarse intermediate frame and aligns the HR features for residual reconstruction. Experiments across spatial and temporal upsampling settings demonstrate that the proposed framework offers a favorable quality--efficiency trade-off.