Following Motion for Sequential Modeling in Video Frame Interpolation

2026-08-24Computer Vision and Pattern Recognition

Computer Vision and Pattern Recognition
AI summary

The authors identify that previous state space models for video frame interpolation struggle with capturing complex motion because they process input in a fixed order. They propose MGMVFI, which uses motion information from optical flow to guide the order of processing, making it better at handling dynamic movements. To deal with errors in motion estimation, the method also uses information from nearby areas to improve feature synthesis. Their approach shows improved performance on standard tests, especially for videos with complicated motion.

State Space ModelsVideo Frame InterpolationOptical FlowMotion-Guided SerializationSequence ModelingFeature PropagationContextual SynthesisDynamic MotionRefinement Block
Authors
Jaehyun Park, Nam Ik Cho
Abstract
State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI), as they can capture long-range dependencies with linear computational complexity. However, their predefined scanning order limits their effectiveness in modeling the dynamic motion trajectories inherent in VFI problems. To tackle this challenge, we propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI. MGMVFI introduces Motion-Guided Serialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state updates with semantically related tokens, enabling motion-consistent feature propagation, particularly for large and dynamic motions. Additionally, to mitigate the unreliable feature representations caused by inaccurate optical flow estimates, we introduce contextual synthesis that utilizes the surrounding spatial context for robust inter-frame feature synthesis. These components are seamlessly integrated within our tailored Mamba architecture, which also employs a lightweight refinement block to enhance local detail reconstruction at a reduced computational cost. Extensive experiments on standard VFI benchmarks demonstrate that MGMVFI achievesstate-of-the-artperformance,particularly on complex and dynamic motions, thereby establishing a new direction for sequence modeling in video interpolation.