Event cameras help drones and ground robots locate each other in tough conditions
EVPeriscope: Extended Perception across Aerial and Ground Vehicles with Event-based Propeller Tracking
Robotics
Summary
It can be hard for flying and ground robots to know exactly where they are relative to each other, especially if cameras get blurry or lose light. The authors developed a special system that uses a fast type of camera to spot the spinning parts of a drone from a robot on the ground. This helps the two robots work together even when it’s dark, windy, or there are lots of plants blocking the view. The drone can then act like a set of extra eyes for the ground robot when its own sensors don’t work well. This system runs quickly and fully on the drone using its own sensors and computer.
event-based camerarelative localizationquadrotor droneground robotmotion blurfiducial markersclosed-loop navigationsensor occlusionpropeller trackingheterogeneous robot teams
Authors
Dexter Ong, Vijay Kumar, Pratik Chaudhari
Abstract
Reliable relative localization between aerial and ground robots is a key requirement for tightly coordinated heterogeneous teams. This can be difficult to do using conventional frame-based cameras and fiducial markers because they are sensitive to motion blur, lighting variations, and payload constraints. This paper presents EVPeriscope, an event-based perception system that enables detection, localization and control of a quadrotor using an upward-facing event camera on a ground robot by detecting the high-frequency visual signature of its propellers. This system allows the quadrotor to function as an extended perception system for the ground robot when onboard sensors exhibit degradation or occlusion. We demonstrate the capabilities of this marsupial ground-aerial system via experiments in challenging field conditions with wind speeds of up to 15 mph, in both daylight and at night. We show that the system supports localization and closed-loop navigation through dense foliage where the ground robot's sensors are occluded. Our control system for the quadrotor operates at 200 Hz entirely with onboard sensing and computation. More details and experiment videos can be found on the project page: https://ongdexter.github.io/evperiscope.