HyperDrone enables ultra-fast realistic drone simulation on GPUs
HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second
RoboticsArtificial IntelligenceMachine Learning
Summary
Simulating drones and their cameras realistically is usually slow, making it hard to train real-world flying skills in a virtual setting. The authors created HyperDrone, a system that runs thousands or millions of drone simulation frames per second by doing physics and advanced 3D rendering entirely on graphics cards. This speed lets them train a quadrotor to control its rotation using only camera views in simulation and then make it work on a real tiny drone with its own camera. Their system makes realistic drone simulation much faster and practical for real robot training.
What this means in practice
- •For drone software developers: Train drone flight control algorithms in highly realistic simulation at unprecedented speeds to improve real-world performance without extensive drone testing.
- •For real-time graphics programmers: Utilize GPU-accelerated ray tracing for fast rendering of complex 3D scenes with physics simulated on the GPU, enabling new visualization and simulation tools.
Authors
Jonas Eschmann, Giuseppe Loianno
Abstract
We present HyperDrone, an extremely fast quadrotor simulator focused on high-performance batch rendering that is more than 20x faster than existing state-of-the-art batch quadrotor simulators. HyperDrone combines highly optimized multirotor dynamics with HyperLux, a new ray-tracing engine that supports Nvidia OptiX, Vulkan, Apple Metal and WebGPU backends that takes full advantage of modern ray-tracing hardware, capable of running both components entirely on the GPU. HyperLux renders RGB, depth, semantic/instance segmentation, surface normals and optical flow observations at configurable levels of fidelity, including reflections, realistic lighting, motion blur, anti-aliasing, and Physically Based Rendering (PBR). Even in the highest-fidelity settings, HyperLux achieves millions of Frames Per Second (FPS) on consumer-grade GPUs while rendering realistic 3D scenes. We demonstrate the merits of HyperDrone by training a vision-based yaw-control policy purely in simulation and transferring it to a real small-scale quadrotor (Crazyflie) with a camera, running completely onboard.