Cyclops: LiDAR as a Camera That Dreams in Color
2026-08-17 • Robotics
Robotics
AI summaryⓘ
The authors created a system called Cyclops that turns sparse LiDAR data, which is usually hard to use like a camera, into dense RGB video images. This lets robots see well even in very dark or tricky lighting without needing actual cameras. They do this by first making the sparse data dense, then converting it to look like regular color images using a special technique involving math and attention to previous frames to avoid flickering. Their experiments show that these generated images help usual camera-based AI models work better than before on tasks like identifying objects, lanes, and coloring point clouds in tough lighting. Overall, their work bridges the gap between LiDAR sensing and traditional vision models.
LiDARRGB video synthesisNon-Repetitive Scanning LiDAR (NRS-LiDAR)Densification moduleLatent Bridge Matching (LBM)Temporal attentionOptical flowSemantic segmentationLane detectionPoint cloud colorization
Authors
Wei Gao, Jian Shu, Mingle Zhao, Maani Ghaffari, David Kong, Chengzhong Xu, Hui Kong
Abstract
Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.