BodyBridge improves realistic pedestrian pose in simulated driving videos
BodyBridge: Grounded Body Layers for Sim-to-Real Translation of Vulnerable Road Users
Computer Vision and Pattern Recognition
Summary
Simulated driving videos often show pedestrians and cyclists as simple shapes, which misses important details like body pose and identity that affect safety. The authors created BodyBridge, a method that connects each pixel to a detailed body model and exact masks, helping computer models generate more accurate images of these road users. This method improves how well simulated videos match real ones, notably reducing errors like body flips and mistaken poses. Their work helps make self-driving car systems better at recognizing and predicting the movements of vulnerable road users.
What this means in practice
- •For autonomous vehicle developers: Improve simulation-based training with more accurate pedestrian and cyclist appearances for safer self-driving systems.
- •For virtual environment designers: Generate realistic human poses in simulated outdoor scenes using explicit body alignment techniques.
Authors
Mark Azer, Fabian B. Flohr
Abstract
Simulation-to-real driving video diffusion models can translate photorealistic scenes, but they typically represent pedestrians and cyclists, vulnerable road users (VRUs), as generic object classes. This is insufficient for vulnerable road users whose safety-relevant appearance depends on articulated pose and identity. We present BodyBridge and Grounded Body Layers (GBL), a simulator-derived, pixel-aligned representation that combines scene geometry, canonical per-pixel body correspondence, and exact VRU masks. GBL provides explicit articulation to the video diffusion model, emphasizes VRU regions during control-branch post-training, and supports additive composition with scene-geometry control at inference. It also enables per-joint evaluation of generated VRUs and a failure taxonomy for identity errors. On CARLA-to-real translation, BodyBridge improves near-range PCK@0.5 by 22% over the released depth-conditioned baseline. Replacing normal-map body control with canonical correspondence reduces body flips by 18%, while composition reduces near-range facing flips by 27%. These results show that explicit body grounding complements scene-level control and improves pose-faithful VRU generation. All code, data, and model checkpoints will be made publicly available to support reproducible, pedestrian-aware autonomous driving research.