Joint forecasting improves 3D motion and path prediction in sports
Unified Trajectory and 3D Pose Forecasting for Multi-Agent Sports Analysis
Computer Vision and Pattern Recognition
Summary
Predicting how people will move in the future is hard because it involves knowing both where they will go and how their bodies will pose. This is especially tricky in team sports where players interact closely. The authors combined information about player paths and body poses into a single system that understands their relationship. They also improved training methods to focus on moving body parts and smoothed the path predictions. Their approach works better than past methods on standard sports and motion datasets.
What this means in practice
- •For sports analytics teams: Enhance the prediction of player movements and poses to improve game analysis and strategy development in team sports.
- •For robotics engineers: Improve robot anticipation of human actions by jointly forecasting trajectories and poses of multiple interacting people.
Authors
Marc Peral, Guillem Capellera, Luis Ferraz, Antonio Rubio, Stella Graßhof, Dan Witzner Hansen, Antonio Agudo
Abstract
Forecasting future human motion requires jointly reasoning about an agent's global trajectory and 3D body pose, which are tightly coupled, particularly in team sports where multiple interacting players introduce complex social dynamics. We exploit a socio-temporal encoder that fuses trajectory and pose at the input level and models them jointly through shared factorized attention. The model is further enhanced by a per-joint loss reweighting scheme that emphasizes dynamic joints and prevents degenerate solutions, and a frequency-domain trajectory smoothing term that improves trajectory prediction while indirectly benefiting pose forecasting. To ensure fair comparison with prior joint forecasting methods, we identify and correct evaluation biases arising from overlapping sequence construction, adopting instead a non-overlapping protocol and a deterministic, oracle-free strategy for multi-future prediction. Our approach improves over prior joint forecasting methods on both WorldPose and 3DPW, with the largest gains on pose accuracy, demonstrating that the proposed joint formulation generalizes beyond the densely interactive scenarios that motivate it.