Latent dataset distillation improves human motion prediction models
Latent Dataset Distillation for Human Motion Prediction
Computer Vision and Pattern Recognition
Summary
Human motion prediction tries to guess future body movements based on past ones, but current methods struggle with making realistic and stable predictions using compressed data. The authors propose a new way to compress motion data by first turning it into a simpler code, then learning from that instead of the original complex motions. This method uses a pretrained decoder to keep predictions realistic and allows gradual improvement through multiple steps. Tests on common motion datasets show this approach beats older methods in accuracy and produces more plausible movements.
What this means in practice
- •For animation studios: Generate more realistic and stable character motions by training models on compact, high-quality synthetic motion datasets derived with latent distillation.$Commercial implications: Provides a way to create smaller motion datasets that enhance animation quality and efficiency for studios that develop realistic character movements.
- •For motion capture system developers: Produce lightweight synthetic motion datasets that improve human motion prediction models used in motion capture data processing and compression.
Authors
Ge Tian, Guang Li, Takahiro Ogawa, Miki Haseyama
Abstract
Dataset distillation (DD) compresses a large training set into a compact synthetic set while preserving downstream training utility. Although DD has been widely studied for images and recently extended to time-series forecasting, its application to human motion prediction remains largely unexplored. Human motion is high-dimensional and structurally coupled, and gradient matching (GM) in the original motion space optimizes many correlated variables without a prior on pose plausibility or temporal dynamics, which frequently yields implausible and unstable synthetic motions. To address this limitation, we propose a latent DD framework that regularizes distillation with a learned motion prior. Motions are first compressed by a residual-quantized variational autoencoder (RVQ-VAE), and distillation then updates only a learnable latent bank through the frozen quantizer and decoder. The pretrained decoder restricts synthetic motions to its output space, while residual quantization progressively refines the latent approximation across multiple codebooks and alleviates the representational bottleneck of single-stage vector quantization. Experiments on Human3.6M, CMU, and 3DPW with two prediction backbones show that the proposed framework outperforms direct GM in 27 of 30 evaluated settings and random subsets in every setting, and produces visibly more plausible synthetic motions in qualitative comparisons.