Camera scale mismatch corrected for stable multi-person 3d human tracking
A Diagnostic Study of World-Space Human Recovery Under High Camera Dynamics
Computer Vision and Pattern Recognition
Summary
Videos with quick, large camera movements often confuse systems trying to find people’s exact locations and poses in the 3D space of a scene. The authors studied why these errors happen in detailed dance videos with many performers and fast camera moves. They found the problem is that the camera and human positions are estimated independently and then combined at different scales, causing the performers’ positions to drift wrongly over time. The authors created a formula that fixes this scale mismatch after the fact, making the recovered positions stable and accurate without needing to retrain the system.
What this means in practice
- •For video production teams: Create stable 3D reconstructions of performers in videos with fast camera moves for virtual editing and reshooting.
- •For computer graphics developers: Generate accurate multi-person choreography data from broadcast videos despite rapid camera dynamics.
Authors
Hao-Jen Chien, Wei-Lun Chao, Yu-Lun Liu
Abstract
Monocular world-space human recovery methods excel on current benchmarks, but these rarely include fast camera motion. We introduce the K-Pop stress test, broadcast dance videos that combine fast, large camera motion with dense multi-person choreography. Leading methods produce plausible per-frame meshes on these videos, yet their world-space trajectories drift far from the performers' stage positions. Through a systematic diagnosis, we show that the camera and human estimates each remain self-consistent, and that the drift appears when they are combined at mismatched scales. Each is recovered at its own scale, so a mismatch between the two can turn camera motion into human drift that grows with camera displacement and becomes severe in broadcast dance videos. We derive a closed-form correction that estimates this mismatch and applies one scale to all performers, removing the drift while keeping their relative positions in one shared world frame. It requires no retraining and works with existing backends. Combined with physical constraints, the correction yields stable world-space reconstruction for choreography modeling, scene editing, data generation, and virtual reshooting.