Geometry aware modeling improves few shot action recognition accuracy
Geometry-Aware Trajectory State Modeling for Few-Shot Action Recognition
Computer Vision and Pattern Recognition
Summary
Recognizing human actions from only a few video examples is hard because motions vary a lot. The authors think of an action as how points move and relate in space and time, using a 2.5D geometric approach. Their new method, GeoTRAM, models motion along trajectories and how these points interact locally over time. They show this helps recognize actions better, especially for those focused on motion, without losing accuracy where appearance matters more.
What this means in practice
- •For video analytics engineers: Improve recognition of human actions from limited labeled videos by modeling motion trajectories with local geometric context.
- •For robotics perception teams: Use geometry-aware trajectory modeling to enhance robots’ ability to identify human actions in dynamic environments with few training examples.
Authors
Zhiqi Miao, Marko Haralović, Stanislav Levendeev, Sarvesh Angadi, Jannatul Ferdous, Dan-Cristian Ploesteanu, Jiapan Guo, Estefanía Talavera
Abstract
Few-shot action recognition (FSAR) requires representations that expose discriminative motion dynamics from only a few labeled videos. Recent trajectory-based methods leverage long-term point tracks for explicit temporal correspondence, but often treat trajectories as carriers of sampled visual features or summarize their motion with compact 2D displacement-based descriptors. We instead view an action as the ordered evolution of 2.5D trajectory states and their local geometric interactions. Based on this view, we introduce GeoTRAM, a geometry-aware trajectory state modeling framework for FSAR. GeoTRAM lifts sparse tracks into 2.5D trajectories, where StateMotion models ordered intra-trajectory state evolution and RelMotion captures state-conditioned local interactions through time-varying relative geometry and motion. Quality-filtered spectral sampling provides more reliable trajectory initialization; complementary semantic, motion, and depth cues are separately refined and preserved for joint spatio-temporal modeling; action-guided reweighting and top-$ρ$ trajectory selection support complementary frame- and trajectory-level matching. Experiments on five FSAR benchmarks demonstrate strong gains on the motion-centric Something-Something benchmarks while maintaining competitive performance on more appearance-driven benchmarks and a favorable accuracy--efficiency trade-off.