Low-light pedestrian detection improved using thermal and RGB camera pairs
Stereo LWIR-RGB Automatic Labeling for Low-Light Camera-Based VRU Detection
Computer Vision and Pattern Recognition
Summary
Detecting pedestrians and cyclists at night is important for safety but hard because normal cameras don’t work well in the dark. The authors developed a way to automatically label images by combining thermal (heat) and regular camera pictures, using the heat to find people even in low light. They made this method cheaper and easier by simplifying the camera setup. When tested, their method created better training data for detection programs than manual labeling. This approach could help improve pedestrian detection systems for nighttime safety.
What this means in practice
- •For automotive safety teams: Build improved pedestrian and cyclist detection systems for night driving using automated labeling from thermal and RGB camera pairs.$Commercial implications: Enables production of more accurate night-time VRU detection features for vehicles by providing affordable, scalable labeling methods.
- •For smart city planners: Deploy low-cost sensor setups combining thermal and RGB cameras to automate detection of vulnerable road users in poorly lit urban areas.
Tested on one dataset.
Authors
Dimitrios Bouzoulas, Pyry Jäppinen, Pejman Habibiroudkenar, Eerik Alamikkotervo, Risto Ojala
Abstract
VRU detection in low-light conditions is a key component in ensuring VRU safety during the most collision prone time of the day. Currently, advances in low-light detection are limited due to the lack of large scale datasets and automated labeling methods. Most autolabeling methods use RGB detections as their labels source, which cannot be replicated in low-light conditions. To fill this gap in this paper we present an automated pipeline for low-light VRU labeling. We use LWIR-RGB image pairs and source our labels from LWIR as VRUs differentiate from the background due to their body temperature. Unlike our previous work that relied on a complex hardware setup to align the two modalities, we perform this experiment using a LWIR-RGB stereo setup, lowering hardware complexity and cost, making the pipeline easier to reproduce. We evaluate method end-to-end on a dataset collected in the capital region of Finland. Models trained on our generated labels beat both off-the-shelf baselines and models fine tuned on a hand labeled subset on F1, mAP@50 and LAMR metrics when tested on a manually labeled dataset, and the result holds for two different LWIR detectors. This shows that low-light VRU autolabeling can be made both accessible and effective without specialized alignment hardware.