HERCULES: An Open-Source Simulation Framework for Heterogeneous Multi-Robot SLAM, Collaborative Perception, and Exploration
2026-06-22 • Robotics
RoboticsComputer Vision and Pattern RecognitionMultiagent Systems
AI summaryⓘ
The authors developed HERCULES, a simulator that helps test and improve groups of flying and ground robots working together in realistic and large environments. It builds on existing simulators but fixes key issues to allow different types of robots to operate at the same time, using shared tools for navigation and control. HERCULES adds new sensors, like infrared cameras for night vision, and can simulate things like fire or traffic to make testing more realistic. It can run by replaying recorded robot paths or by letting robots plan their actions live. The authors also provide their code and datasets, including a challenge for robot mapping using multiple drones and ground robots in various terrains.
multi-robot autonomyUAVUGVSLAMnavigation stackUnreal Engine 5infrared camerasrobot simulationROS 2photorealistic environments
Authors
Sandilya Sai Garimella, Daniel Chase Butterfield, Sean Wilson, Lu Gan
Abstract
We present HERCULES, an open-source simulator and data-collection pipeline for heterogeneous multi-robot autonomy. Built upon the Unreal Engine 5 (UE5)-based simulators AirSim and Cosys-AirSim, HERCULES resolves key architectural limitations of prior frameworks to enable concurrent unmanned aerial and ground vehicle (UAV-UGV) operation in large-scale, photorealistic, dynamic environments. It introduces a new waypoint-tracking UGV controller that mirrors existing UAV control interfaces, and provides a shared navigation stack for mapping, traversability analysis, planning, and control across heterogeneous platforms. Expanding inherited sensor suites, it adds physics-based long-wave infrared (LWIR) cameras and configurable night-vision modes for degraded visual environments. HERCULES provides lightweight APIs, ROS 2 wrappers, and rigorous time synchronization across sensors and platforms, and brings state-of-the-art game-engine capabilities into robotics simulation, integrating intelligent agents such as pedestrians, traffic, and wildlife with high-fidelity dynamic phenomena, including fire, flooding, and crop disease spread. HERCULES runs in two modes: passively, replaying offline-designed trajectories to generate reproducible multi-modal datasets, and actively, running an online planner in closed loop from live observations. Our experiments in heterogeneous multi-robot SLAM, collaborative perception, and exploration, using both HERCULES-generated data and active closed-loop execution, demonstrate its utility for advancing heterogeneous multi-robot autonomy. We publicly release our source code, experiment code, documentation, and datasets, including a heterogeneous multi-robot SLAM benchmark collected with two UAVs and two UGVs across kilometer-scale desert, forest, and city environments, at https://lunarlab-gatech.github.io/HERCULES-website.