WildDepth 2 dataset combines RGB LiDAR and radar for wildlife tracking
WildDepth-v2: A Synchronized RGB--LiDAR--Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife
Computer Vision and Pattern Recognition
Summary
Measuring how far animals are and how they move is important but hard because regular cameras can't measure distance in meters. The authors created WildDepth 2, a dataset that combines regular video, LiDAR, and radar recordings of wildlife, including some unique radar data. This lets computers estimate the real-world positions and movements of animals more accurately. Their work also shows how radar can improve depth and motion estimates even when LiDAR isn't used all the time.
What this means in practice
- •For wildlife monitoring teams: Improve animal distance and motion measurement by using synchronized RGB, LiDAR, and radar data for better population and movement studies.
- •For autonomous vehicle developers: Enhance detection and tracking of animals near roads using combined radar and LiDAR sensor fusion informed by realistic wildlife datasets.
Authors
Muhammad Aamir, Sangyun Shin, Naoya Muramatsu, Matthew Wijers, Amir Patel, Andrew Loveridge, Andrew Markham
Abstract
Measuring animal distance and motion in metric units underpins abundance estimation, morphometrics, and movement ecology, yet monocular video lacks metric scale and wildlife datasets lack metric ground truth. We present WildDepth 2, extending the synchronized RGB-LiDAR WildDepth dataset in three directions. (i) Imaging radar: 37 RGB-LiDAR-4D-radar sequences (19,716 frames) with per-point Doppler velocity and a measured error budget, to our knowledge the first imaging-radar wildlife recordings. (ii) Benchmarking: a frozen, checksummed hidden test set for RGB-only distance estimation and RGB+sparse-LiDAR depth completion, evaluated separately on animal and background pixels. (iii) Motion and tracking: metric 3D trajectories obtained by lifting 2D tracks using LiDAR, radar, or monocular depth. Across 48,723 animal frames, object-level radar calibration improves monocular depth over scene-level LiDAR calibration without requiring LiDAR at inference (AbsRel 0.120 vs. 0.198), although animal-level LiDAR remains more accurate (0.052). Across 431 tracks, once-per-track radar scaling reduces 3D position error four-fold (2.20 to 0.58 m) and distance-sampling detection-function bias from -13% to -2%, matching once-per-track LiDAR calibration. Transferring radar Doppler onto LiDAR produces motion masks 3.7 times denser than radar alone at 72% precision, reduces moving-animal LiDAR accumulation error by 37%, and estimates radial speed (r = 0.60), albeit with approximately 50% underestimation.