Event based optical flow improves motion tracking in vr ar
E-WAVE: Event-based Continuous Optical Flow via Warping-Aligned Visual Encoding
Computer Vision and Pattern Recognition
Summary
Capturing fast motion in virtual and augmented reality is tricky because traditional cameras can only process images at fixed speeds. The authors propose E-WAVE, a method using event cameras that capture changes very quickly and efficiently. Instead of heavy computations comparing pairs of points, E-WAVE uses a technique that estimates motion along smooth paths and can predict movement at any moment without repeating calculations. Tests show it tracks motion more accurately and works well even in real-world scenarios with a wearable device.
What this means in practice
- •For vr ar developers: Improve fast and continuous motion tracking of hands and objects in immersive VR/AR systems using event-based optical flow without heavy computation.
- •For robotics engineers: Enhance robots' perception of rapid movements in dynamic environments by integrating event-camera based flow estimation that works efficiently over time.
Authors
Jiale Wu, Xiaoyang Bai, Haoming Yu, Yiwei Chen, Yifan Peng, Weiwei Xu
Abstract
Temporally dense optical flow is essential for dynamic perception in immersive VR/AR systems, where rapid head, hand, and object motion must be continuously captured and tracked. Existing frame-based optical flow estimation methods are constrained by the tradeoff between temporal resolution and computational cost; while event cameras, with their high temporal resolution and energy efficiency, serve as a natural solution to the dilemma. However, event-based approaches commonly rely on correlation volumes to capture pairwise voxel correspondences, which incur substantial memory and computation overhead. We present E-WAVE, a correlation-free framework for high-temporal-resolution (HTR) optical flow estimation from event streams. Instead of constructing all-pairs correlation volumes, E-WAVE employs global attention mechanism to model long-range feature dependencies and performs trajectory guided feature warping using Bézier curve. Through iterative updates, it predicts trajectories that allow for querying at arbitrary timestamps without repeated inference. Experiments on MultiFlow and DSEC-Flow demonstrate a 25% lower trajectory error and comparable endpoint flow estimation accuracy relative to state-of-the art baselines. Additional evaluations on self-captured data using a head-mounted prototype validate that E-WAVE remains robust under challenging real-world conditions.