Carlaverse creates immersive simulation for safe human driving tests
CARLAverse: A Highly Modular, Distributed, and Multimodal Framework for Human-in-the-Loop Simulation
RoboticsHuman-Computer Interaction
Summary
Testing self-driving cars in real life can be risky, especially around pedestrians and cyclists. The authors created Carlaverse, a simulation where humans can safely control virtual cars, bikes, or walk as pedestrians all together in one shared environment. It uses a clever system to reduce network delays, so the virtual experience feels realistic and responsive. This setup helps researchers study human behavior with autonomous vehicles better than before.
What this means in practice
- •For autonomous vehicle developers: Run large-scale human-in-the-loop simulations combining drivers, cyclists, and pedestrians while preserving realistic physical feedback.
- •For virtual reality hardware designers: Test and improve haptic devices for realistic driving experiences by integrating latency-sensitive local physics with global traffic control.
Authors
Patrick Rebling, Philipp Nenninger, Reiner Kriesten
Abstract
The development of autonomous driving demands comprehensive testing in mixed-traffic scenarios involving vulnerable road users (VRUs), where purely artificial agents often fail to capture authentic human social negotiations. While human-in-the-loop (HITL) simulators enable safe investigation of these interactions, existing multi-agent platforms struggle with the network latency and synchronization constraints required for high-fidelity haptic feedback. To resolve this, we present CARLAverse, an open-source, multimodal simulation ecosystem. Extending modular hardware abstraction, CARLAverse integrates driving (DrivoCARLA), cycling (CycloCARLA), and pedestrian (WalkoCARLA) simulators into a shared virtual environment. Its core methodological contribution is a distributed physics architecture: latency-critical ego dynamics and high-frequency force feedback are computed locally on client nodes, while a central CARLA server orchestrates non-player character (NPC) physics and global traffic. By decoupling haptic control loops from network bottlenecks, CARLAverse enables scalable, cross-institutional HITL experiments without compromising physical immersion. Code and documentation: https://git.ieem-ka.de/simulator-environments/carlaverse