Local temporal contrasts reveal geometric patterns in human motion data
Local Temporal Contrasts on Human Motion Data Induce a Latent Geometry
Machine Learning
Summary
Human motion data involves complex movement patterns that happen at different speeds and times. Existing methods often simplify these patterns into single points, losing important information about how movements change over time. The authors developed a new method called SemTAC that keeps track of how motions evolve, showing meaningful curves and shapes in the data. Their approach makes it easier to understand and predict motion, especially when only partial movement sequences are available.
What this means in practice
- •For human motion analysis teams: Improve recognition and prediction of incomplete human motion sequences in applications like sports analytics or physical therapy.
- •For motion capture system developers: Enhance motion encoding by capturing geometric latent states for better downstream motion understanding and synthesis.
Authors
Lourenço A. Rodrigues, Felix Putze
Abstract
Time series such as recordings of human motion data often encode structure across multiple temporal scales, and when a latent state determining this sequence evolves, this structure needs to survive in the representation space for downstream use. Yet the currently prevalent attention-based encoders paired with sequence-level pooling have made single-vector, pointwise comparison the default mode of inference, discarding the trajectory a sequence traces through the representation space. We propose recovering this geometric structure directly, without relying on forecasting-based objectives, by training an encoder model (SemTAC) with a temporal-neighborhood contrastive loss: representations that satisfy local consistency everywhere must give rise to a coherent global structure, letting inference happen at temporal scales beyond those seen during training. We evaluate SemTAC against four baselines on a five-step motion sequence discrimination task: the same architecture trained with a classification or a standard contrastive objective, and two established self-supervised time-series methods, TS-TCC and TS2Vec. At the full sequence length all five are broadly competitive, but on incomplete, truncated sequences SemTAC holds a large advantage: a median F1 of 0.77 one step before the sequence completes, against 0.27-0.70 for the four baselines. A hyperparameter-matched ablation attributes this effect specifically to the temporal-contrastive objective. SemTAC's representation trajectories are also the only ones among these five to exhibit interpretable geometric structures, exceeding a no-structure chance baseline on a displacement-smoothness metric which we introduce. This is evidence that its objective, not classification accuracy alone, produces genuinely temporally-ordered trajectories.