MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations
2026-06-22 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors propose MotionMAR, a method to reconstruct detailed human motion from limited data by working from broad movements down to finer details. Their approach breaks down motion into different time scales, first capturing general movement patterns and then adding detailed corrections. They use several interconnected modules to encode, predict, control, and refine the motion data, ensuring it matches real observations and looks smooth. Experiments show their method performs very well on a standard human motion dataset.
Human motionAutoregressive modelTemporal hierarchyVQ-VAELatent spaceMotion reconstructionSparse observationsTemporal multi-scaleQuantization artifactsAMASS dataset
Authors
Yuhua Luo, Junsheng Zhang, Mengyin Liu, Xincheng Lin, Ming Yan, Zhudi Chen, Chenglu Wen, Lan Xu, Siqi Shen, Cheng Wang
Abstract
Human motion follows a temporal hierarchical structure, transitioning from low-frequency global trajectories to high-frequency details. Inspired by the success of multi-level autoregressive models in computer vision, we propose MotionMAR, a coarse-to-fine framework for motion reconstruction from sparse observations. It first estimates the global trajectory of human motion and then gradually refines the temporal details. This architecture consists of four integrated components. The Temporal Multi-scale Tokenization (TMT) VQ-VAE encodes the data at multiple temporal resolutions, separating semantic motion from minor jitters. The Motion Autoregressive Network (MAN) operates in this latent space, predicting motion across scales. It first establishes the global structure through coarse indices and then generates finer indices to recover specific details. Meanwhile, the Scale-Aware Control (SAC) module integrates sparse tracking data to ensure the generated output aligns with actual observations. The Motion Refinement Network (MRN) subsequently smooths consecutive poses and eliminates quantization artifacts. Experiments show that MotionMAR achieves state-of-the-art accuracy on the AMASS dataset, providing a reliable and structure-aware approach for motion reconstruction. The source code is publicly available at http://www.lidarhumanmotion.net/motionmar/.