GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking
Computer Vision and Pattern Recognition
Summary
The authors study how to track moving features in videos using event cameras, which capture motion very quickly and efficiently. Instead of using fixed rules to decide how to process camera data, they teach a computer program using reinforcement learning to adaptively choose when to gather data and when to analyze it. This approach helps the system handle different types of motion better, especially fast or sudden movements. They also created a new dataset to test their method and showed that it improves tracking accuracy and efficiency compared to older fixed-rule methods.
Authors
Youngho Kim, Hoonhee Cho, Jae-Young Kang, Kuk-Jin Yoon
Abstract
Feature tracking plays a fundamental role in understanding scene motion and supports various downstream tasks. Event cameras, with their high temporal resolution and asynchronous sensing, enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion. However, existing event-based feature tracking methods rely on fixed heuristic rules based on hand-tuning for event accumulation. Such strategies fail to adapt to diverse motion dynamics, leading to degraded performance under abrupt motion changes or low-motion scenarios. In this paper, we model event accumulation as a sequential decision-making problem and introduce reinforcement learning (RL) framework to adaptively control the accumulation process for online event-based feature tracking. Our approach trains a RL agent that decides whether to continue accumulating events or to perform tracking inference based on motion cues. The proposed adaptive temporal agent enables dynamic adaptation to varying motion patterns without relying on hand-crafted rules. Furthermore, we introduce a Dynamic Event-based Tracking (DEFT) dataset with dynamic motion distributions to evaluate the robustness of the feature tracking. Extensive experiments demonstrate that integrating our plug-and-play framework to existing feature tracking methods consistently outperforms heuristic-based approaches, improving robustness under dynamic motion while offering a better balance between tracking accuracy and efficiency. Our project codes and datasets are available at https://github.com/kmax2001/GoSTOP