CrowdTraj benchmark reveals challenges in dense crowd trajectory prediction
CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments
Computer Vision and Pattern Recognition
Summary
CrowdTraj is a new dataset that helps test how well computers can predict where people will move in very crowded spaces, using real security camera footage. Earlier tests mostly looked at simpler cases with fewer people or perfect tracking, which doesn’t happen in real life. This dataset includes very busy scenes with lots of occlusions and sudden movements, making predictions harder. The authors found that current methods struggle more with this complexity and need more computing power to learn from such crowded examples.
What this means in practice
- •For public safety teams: Improve monitoring systems by assessing how well pedestrian movement can be predicted in high-density crowded areas using real CCTV footage.
- •For urban planners: Design better pedestrian infrastructure by analyzing natural crowd behaviors and movements in realistic, densely populated environments.
Authors
Antonius Bima Murti Wijaya, Paul Henderson, Marwa Mahmoud
Abstract
In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benchmarks either contain relatively sparse pedestrian interactions, assume perfect tracking inputs, or rely on overhead viewpoints that minimize occlusion and perspective distortion, limiting evaluation in realistic dense-crowd scenarios. We present CrowdTraj, a benchmark for pedestrian trajectory prediction in natural dense crowd scenes. Unlike previous datasets, CrowdTraj supports end-to-end evaluation from detection through tracking to trajectory prediction under severe occlusion in CCTV views. It also captures diverse, natural pedestrian behaviours, including abrupt directional changes rarely observed in existing benchmarks. CrowdTraj includes five diverse scenes, with an average of 1,146 unique pedestrians per scene, maximum frame-level densities ranging from 114 to 372 pedestrians, and over 3.2 million annotated head bounding boxes. CrowdTraj provides pixel and real-world coordinates via per-scene homography matrices for physically meaningful analysis. Our experimental results show that tracking accuracy (IDF1) drops to 0.68 to 0.70 in the densest scenes, compared with approximately 0.90 in less crowded scenes. Trajectory prediction training also becomes substantially more computationally expensive in dense scenes, with training times increasing by up to 8 times. These findings show that CrowdTraj exposes limitations in current trajectory prediction pipelines that remain hidden on existing sparse-crowd benchmarks, particularly in robustness to tracking noise and computational scalability.