Improved road scene segmentation with RGB and near infrared images
HSI-Road Relabeled: Surface-Aware Road-Scene Segmentation
Computer Vision and Pattern Recognition
Summary
It can be hard for computer programs to tell different road surfaces apart, like asphalt, concrete, or water. This paper adds new detailed labels to a dataset that has regular camera images and special near-infrared images. The authors tested models using different types of input images and found that using both registered RGB and near-infrared images together helps recognize surfaces better at similar image resolutions. They also show that image resolution is important for accurate results.
What this means in practice
- •For autonomous vehicle engineers: Improve road surface detection by combining registered RGB and near-infrared images, enabling better semantic segmentation under limited image resolution conditions.
- •For urban planners: Obtain detailed surface-type maps of road scenes by using models trained on the labeled RGB-NIR dataset, aiding infrastructure monitoring and maintenance decisions.
Tested on one dataset.
Authors
Imad Ali Shah, Imran Mehmood, Enda Ward, Martin Glavin, Edward Jones, Brian Deegan
Abstract
The HSI-Road dataset provides paired RGB and 25-channel NIR (600--960~nm) images with binary masks but no surface-level labels.~This paper introduces a manually labeled six-class taxonomy: Background, Asphalt, Concrete, Dirt, Water, and Grass, and an RGB-to-NIR registration pipeline with corresponding annotations. Six semantic-segmentation models (SSMs) are evaluated under four input configurations: original-resolution RGB (RGB$_{\text{ori}}$), registered low-resolution RGB (RGB$_{\text{reg}}$), NIR, and channel-stacked RGB$_{\text{reg}}$--NIR (RGBN$_{\text{stk}}$). The comparison quantifies the effect of spatial-resolution reduction on RGB, along with evaluation of NIR and RGBN$_{\text{stk}}$, with results reported using per-class and mean IoU and F1 scores. RGB$_{\text{ori}}$ achieves the highest overall performance but contains 12$\times$ more pixels than the matched-resolution inputs. At the matched 192$\times$384 resolution, RGBN$_{\text{stk}}$ outperforms NIR for all six SSMs and RGB$_{\text{reg}}$ for five of six, with the most consistent gains for the Water class. These results highlight the importance of spatial resolution while showing that NIR provides complementary information to RGB.