Geometry and temporal clues improve 3d soccer player pose in world coordinates

Field Converter: Geometry-Initialized Temporal Residual Refinement for World-Grounded Player Pose Estimation from Soccer Broadcasts

Computer Vision and Pattern Recognition

Summary

Estimating the exact 3D positions and movements of soccer players from regular TV camera feeds is hard because it requires placing players correctly on the soccer field, not just relative to themselves. The authors introduce a method called Field Converter that first uses camera setups and soccer field shapes to guess a player's location, then refines this guess using clues from video frames and the player’s past movements. This approach cuts down location errors significantly compared to just using geometry. They also find that using sequences of frames helps more than the exact kind of processing used, but jumps or players flying can still confuse the system.

What this means in practice

  • For sports video analysts: Generate accurate 3D positions of soccer players in real-world coordinates from broadcast footage to improve performance analysis.
  • For sports video game developers: Create realistic player animations grounded in the actual field positions using broadcast video data enhanced by geometry and temporal refinement.$Commercial implications: Enables selling enhanced game realism products by using broadcast-based 3D player positioning, improving virtual soccer experiences.

Authors

Simon Khan, Laurent Gajny, Jennyfer Lecompte, Sébastien Laporte

Abstract

Recovering 3D human pose from monocular sports broadcasts remains challenging when players must be localized in a shared metric world coordinate system rather than only reconstructed relative to their own body. We introduce Field Converter, a geometry-initialized temporal residual framework for world-grounded 3D player pose estimation from calibrated soccer broadcasts. Our method first uses camera and pitch geometry to initialize the player root through ray-ground intersection, then predicts a temporal residual correction from pose, image, camera, and geometric cues. On match-disjoint evaluation sequences, residual refinement reduces root error from 49cm with geometry alone to 14cm with a frame-wise MLP and 10cm with a TCN, while a Transformer achieves a comparable 11cm. The resulting world-space MPJPE reaches 13.2cm, and ablations show that residual prediction clearly outperforms direct global-root regression while temporal context matters more than the specific temporal backbone. Failure analysis further identifies airborne motion as the main limitation of the ground-based geometric initialization.
View PDFOpen arXiv