Summary
Predicting where people will move in crowded places is important but hard because cameras often miss people or lose track of them when the crowd is dense. The authors created CrowdTraj, a new dataset that shows very crowded scenes captured by regular security cameras where people often change directions suddenly. This dataset helps test how well computer models can detect, follow, and predict pedestrian movements when many people overlap or hide each other. They found that current methods struggle with accuracy and take much longer to learn in these crowded scenarios than in less crowded ones. This shows that real-life crowded environments still pose big challenges for pedestrian movement prediction.
pedestrian trajectory predictiondense crowdstracking accuracyocclusionCCTV viewsdetectioncrowd behaviorhomography matrixcomputational scalabilityIDF1 metric
Abstract
In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benchmarks either contain relatively sparse pedestrian interactions, assume perfect tracking inputs, or rely on overhead viewpoints that minimize occlusion and perspective distortion, limiting evaluation in realistic dense-crowd scenarios. We present CrowdTraj, a benchmark for pedestrian trajectory prediction in natural dense crowd scenes. Unlike previous datasets, CrowdTraj supports end-to-end evaluation from detection through tracking to trajectory prediction under severe occlusion in CCTV views. It also captures diverse, natural pedestrian behaviours, including abrupt directional changes rarely observed in existing benchmarks. CrowdTraj includes five diverse scenes, with an average of 1,146 unique pedestrians per scene, maximum frame-level densities ranging from 114 to 372 pedestrians, and over 3.2 million annotated head bounding boxes. CrowdTraj provides pixel and real-world coordinates via per-scene homography matrices for physically meaningful analysis. Our experimental results show that tracking accuracy (IDF1) drops to 0.68 to 0.70 in the densest scenes, compared with approximately 0.90 in less crowded scenes. Trajectory prediction training also becomes substantially more computationally expensive in dense scenes, with training times increasing by up to 8 times. These findings show that CrowdTraj exposes limitations in current trajectory prediction pipelines that remain hidden on existing sparse-crowd benchmarks, particularly in robustness to tracking noise and computational scalability.