motion capture system developers

Abstract: Time series such as recordings of human motion data often encode structure across multiple temporal scales, and when a latent state determining this sequence evolves, this structure needs to survive in the representation space for downstream use. Yet the currently prevalent attention-based encoders paired with sequence-level pooling have made single-vector, pointwise comparison the default mode of inference, discarding the trajectory a sequence traces through the representation space. We propose recovering this geometric structure directly, without relying on forecasting-based objectives, by training an encoder model (SemTAC) with a temporal-neighborhood contrastive loss: representations that satisfy local consistency everywhere must give rise to a coherent global structure, letting inference happen at temporal scales beyond those seen during training. We evaluate SemTAC against four baselines on a five-step motion sequence discrimination task: the same architecture trained with a classification or a standard contrastive objective, and two established self-supervised time-series methods, TS-TCC and TS2Vec. At the full sequence length all five are broadly competitive, but on incomplete, truncated sequences SemTAC holds a large advantage: a median F1 of 0.77 one step before the sequence completes, against 0.27-0.70 for the four baselines. A hyperparameter-matched ablation attributes this effect specifically to the temporal-contrastive objective. SemTAC's representation trajectories are also the only ones among these five to exhibit interpretable geometric structures, exceeding a no-structure chance baseline on a displacement-smoothness metric which we introduce. This is evidence that its objective, not classification accuracy alone, produces genuinely temporally-ordered trajectories.

Abstract: Skeleton-based human action recognition has gained increasing attention due to the stability of skeleton data across diverse environments and advances in motion capture technologies. Following the success of supervised learning approaches, self-supervised learning approaches, particularly contrastive learning ones, have achieved impressive performance. However, they remain vulnerable to adversarial attacks. In this work, we propose a plug-in robustification module via quantization for contrastive learning in self-supervised skeleton-based human action recognition (QCLR), which enhances the robustness of representations against adversarial attacks while preserving their representational capability. Specifically, we employ a dual-level quantization scheme across the encoding pipeline, which breaks the gradient propagation and absorbs small adversarial perturbations within the quantization bins. To mitigate the impact of inevitable quantization error, we further design an attention-guided multi-faceted quantization strategy to maintain the representation quality. Experiments on several benchmark datasets demonstrate that integrating QCLR into a progressive lineage of mainstream contrastive learning models substantially improves robustness against both gradient-based white-box and gradient-free black-box adversarial attacks, while preserving highly competitive classification accuracy.

Abstract: Dataset distillation (DD) compresses a large training set into a compact synthetic set while preserving downstream training utility. Although DD has been widely studied for images and recently extended to time-series forecasting, its application to human motion prediction remains largely unexplored. Human motion is high-dimensional and structurally coupled, and gradient matching (GM) in the original motion space optimizes many correlated variables without a prior on pose plausibility or temporal dynamics, which frequently yields implausible and unstable synthetic motions. To address this limitation, we propose a latent DD framework that regularizes distillation with a learned motion prior. Motions are first compressed by a residual-quantized variational autoencoder (RVQ-VAE), and distillation then updates only a learnable latent bank through the frozen quantizer and decoder. The pretrained decoder restricts synthetic motions to its output space, while residual quantization progressively refines the latent approximation across multiple codebooks and alleviates the representational bottleneck of single-stage vector quantization. Experiments on Human3.6M, CMU, and 3DPW with two prediction backbones show that the proposed framework outperforms direct GM in 27 of 30 evaluated settings and random subsets in every setting, and produces visibly more plausible synthetic motions in qualitative comparisons.