FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning
2026-08-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on event cameras, which capture quick, sparse video data differently from normal cameras. They note that previous methods to use this data ignored key advantages by converting it into grids or needing extra training data. To fix this, they created FLEET, a new tool that directly processes event streams using smart math tricks to keep the data small and easy to handle. Their tests show that FLEET works better and is more reliable than older methods, especially when the event data speed changes.
event camerasreinforcement learningfeature extractionrandom Fourier featurescross-attentionlatent representationCNNtokenizationasynchronous data
Authors
Tristan Gottwald, Maximilian Schier, Melanie Schaller, Bodo Rosenhahn
Abstract
Event cameras generate asynchronous, high-frequency data streams offering spatially sparse information at lower latency than traditional cameras.In principle, these properties should be ideal for the design of control policies.However, reinforcement learning research in this field remains limited as existing approaches fail to fully exploit the sensor's properties.CNN-based methods negate the sensors benefits by aggregating events into sparse grids. This couples compute cost to sensor resolution and blurs the temporal information. Meanwhile, existing generative baselines rely on the availability of trajectory data to pretrain the model. We propose FLEET (Feature Learning from Events via Efficient Tokenization), a feature extractor that processes event sequences directly. Leveraging random Fourier features and cross-attention, our architecture compresses variable streams into fixed-size latent representations. This decouples inference cost of the feature extractor's backbone from the sensor's resolution, enabling end-to-end learning without auxiliary losses. We validate FLEET on a new, high-throughput benchmark. The results demonstrate that our sequence-based approach surpasses SOTA performance and exhibits superior robustness to variations in observation frequencies.