Nighttime image semantic segmentation with retinex theory

Nighttime image semantic segmentation is challenging due to low-light and diverse lighting conditions. A straightforward solution is to first enhance nighttime scene images to resemble daytime scene before performing segmentation. This kind of methods heavily rely on the enhancement quality. Inspire...

Full description

Saved in:
Bibliographic Details
Published in:Image and vision computing Vol. 148; p. 105149
Main Authors: Sun, Zhichao, Zhu, Huachao, Xiao, Xin, Gu, Yuliang, Xu, Yongchao
Format: Journal Article
Language:English
Published: Elsevier B.V 01.08.2024
Subjects:
Nighttime vision
Retinex decomposition
Semantic segmentation
Semantic segmentation
Retinex decomposition
Nighttime vision
ISSN:0262-8856, 1872-8138
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Nighttime image semantic segmentation is challenging due to low-light and diverse lighting conditions. A straightforward solution is to first enhance nighttime scene images to resemble daytime scene before performing segmentation. This kind of methods heavily rely on the enhancement quality. Inspired by the Retinex theory for low-light image enhancement, which decomposes an image into reflectance and illumination components, we propose a novel nighttime image segmentation method with Retinex theory (RNightSeg). Our core insight is to obtain high-quality illumination-independent reflectance component to enhance segmentation. Specifically, we apply a decomposition decoder to the backbone network for generating the reflectance component. In addition to the fidelity loss and Total Variation loss for the reflectance component regression, we model the brightening illumination component to enhance the nighttime image and apply the color constancy loss on the enhanced image. This helps to cope with the issue of low-light and diverse lighting in the nighttime scene. Finally, we fuse the reflectance decoder feature with the backbone feature and feed the fused feature to the segmentation decoder. Extensive experimental results on two widely used datasets demonstrate that the proposed RNightSeg achieves superior performance over some state-of-the-art segmentation methods. The code of our implementation is available at https://github.com/sunzc-sunny/RNightSeg. •Developed a Retinex-based network RNightSeg to enhance nighttime image segmentation.•Introduced illumination-independent reflectance component for accurate segmentation.•Method improves performance across diverse nighttime illumination conditions.
ISSN:0262-8856
1872-8138
DOI:10.1016/j.imavis.2024.105149