FreeFlow transformer improves optical flow accuracy without bias
FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation
Computer Vision and Pattern Recognition
Summary
Optical flow measures how things move between two images, useful for things like video editing or self-driving cars. Most methods use specific rules to do this accurately, which can make them complicated and limited. The authors created FreeFlow, a model that doesn't rely on these rules but uses a special kind of neural network called a transformer to learn movement directly. Despite ignoring traditional methods, FreeFlow matches or beats top results on popular tests and works efficiently on high-resolution images.
What this means in practice
- •For computer vision engineers: Integrate FreeFlow to estimate motion in video for tasks such as autonomous driving and video stabilization without handcrafted assumptions in the model.
- •For augmented reality developers: Improve real-time motion tracking in AR apps at high resolutions efficiently using FreeFlow's transformer-based optical flow method.
Authors
Vladislav Bargatin, Alexander Yakovenko, Khaled Abud, Dmitriy Vatolin
Abstract
Optical flow methods typically rely on task-specific inductive biases, such as correlation volumes, feature warping, and iterative refinement, among others, to reach high accuracy. While effective, such biases constrain the model to predefined heuristics, which can limit its expressivity and lead to more complex pipelines and additional computational cost. We present FreeFlow, a hierarchical transformer built without any flow-specific components, using instead a single feed-forward encoder--decoder. FreeFlow combines three attention variants: window attention for local processing, shifted-window attention for cross-window information exchange, and a global attention operating at a reduced resolution. The resulting architecture scales naturally with model capacity, enabling a consistent accuracy gain from small to large variants. Despite the absence of standard inductive biases, FreeFlow achieves state-of-the-art results on major benchmarks, including Sintel (0.68/1.48 EPE on Clean/Final), KITTI-2015 (3.23 Fl-all), and Spring (3.192 1px), while remaining memory efficient at 1080p inference.