Simplified Cross-Modal Calibration for Heterogeneous Event-RGB Stereo Systems
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created an easier way to align event-based and regular frame-based cameras without needing the cameras or targets to move. They use a special pattern shown on screens that triggers events while staying visible to normal cameras, making synchronization simpler. Their method cleans up event data, aligns it with regular images, and calibrates the cameras more accurately than previous techniques. They tested it in different lighting and angles and showed it works well for robot vision tasks, even with some parts blocked.
extrinsic calibrationevent-based cameraframe-based cameraChArUco targetstereo calibrationevent-to-image reconstructionrobotic eye-to-hand calibrationreprojection errorsensor synchronizationdenoising
Authors
Nico Hessenthaler, Adam T. Müller, Nicolaj C. Stache
Abstract
Accurate extrinsic calibration between event-based and frame-based cameras remains a practical bottleneck for heterogeneous stereo systems. Existing approaches often require sensor or target motion, precise synchronization, or computationally expensive event-to-image reconstruction. We propose a simple, motion-free cross-modal calibration framework that uses a temporally modulated, blended ChArUco target presented on standard consumer displays. By alternating between the original pattern and a partially blended version, the target reliably triggers events while remaining continuously observable to a frame-based camera, avoiding blank frames and reducing synchronization constraints to a coarse, trigger-based alignment. We discretize events into frames coarsely aligned with the RGB images, apply lightweight denoising, and perform ChArUco-based intrinsic and stereo extrinsic calibration. Extensive experiments assess robustness to blending opacity, display brightness, external illumination, viewing angle, and handheld acquisition. Compared to the strongest motion-based reference (E2Calib + Kalibr) and a non-motion-based reference (Plasberg et al.), our approach reduces the mean reprojection error by $44\%$ and $6\%$, respectively, while substantially simplifying the calibration procedure. Finally, we demonstrate practical utility in a robotic eye-to-hand calibration case study, showing consistent transformations and stable downstream geometric measurements even under partial occlusions. Code is publicly available at https://github.com/nhessenthaler/simple-evrgb-cal.