VARPose: Flexible 2D Pose Densification via Visual Autoregressive Modeling for Enhanced 3D Lifting
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors propose VARPose, a method to add more detailed points to simple 2D human skeleton poses to help with 3D pose tasks. They introduce a Pose Tokenizer that turns poses of different detail levels into a shared code format, and a model called UniSkelar that predicts additional pose points step-by-step from simple to detailed. Their approach improves pose understanding and helps other tasks like 3D pose estimation work better even with poses that have never been seen before. They made their code publicly available for others to use.
Visual AutoRegressive Modelingpose estimation2D human skeleton3D pose liftingtokenizerresidual quantizationautoregressive modelpose densificationmulti-scale representationhuman mesh recovery
Authors
Kaiyuan Pu, Tiantian Yang, Dan Zeng
Abstract
Visual AutoRegressive Modeling (VAR) has excelled in natural image generation via next-scale prediction, but its use on topology-structured data like human skeletons is still unexplored. VARPose is proposed to adaptively densify 2D sparse poses, thereby enriching the anatomical information available for 3D lifting models. Our core contributions are twofold. First, we introduce a Granularity-agnostic Pose Tokenizer (GPT), which employs a single hybrid codebook and a residual quantization strategy to encode poses of varying densities into a unified, multi-scale discrete representation. Our results demonstrate the strong generalizability of this representation. By decoupling the representation from the projection, we can successfully decode novel pose granularities using a frozen codebook with a retrained decoder. Second, we propose UniSkelar, a unified autoregressive model that treats "joint density" as "scale". UniSkelar learns to predict the token sequence for the next density level in a coarse-to-fine manner, conditioned on the sparsest pose. VARPose not only outperforms state-of-the-art methods and generalizes to unseen granularities, but also confers tangible performance gains on downstream tasks, such as 3D Pose Estimation and Human Mesh Recovery, through 2D pose densification. Our code and model are available at https://github.com/BRL-SYSU/VARPose.git.