Instance Segmentation in the Dark

Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. In this work, we take a deep look at instance segmentation in the dark and introduce several techniques that substantiall...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision Vol. 131; no. 8; pp. 2198 - 2218
Main Authors: Chen, Linwei, Fu, Ying, Wei, Kaixuan, Zheng, Dezhi, Heide, Felix
Format: Journal Article
Language:English
Published: New York Springer US 01.08.2023
Springer
Springer Nature B.V
Subjects:
Algorithms
Analysis
Artificial Intelligence
Computer Imaging
Computer Science
Datasets
Feature maps
Image Processing and Computer Vision
Image segmentation
Instance segmentation
Learning
Light
Neural networks
Noise control
Noise reduction
Pattern Recognition
Pattern Recognition and Graphics
Performance degradation
Vision
Instance segmentation
Low-light image dataset
Feature denoising
Object detection
ISSN:0920-5691, 1573-1405
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. In this work, we take a deep look at instance segmentation in the dark and introduce several techniques that substantially boost the low-light inference accuracy. The proposed method is motivated by the observation that noise in low-light images introduces high-frequency disturbances to the feature maps of neural networks, thereby significantly degrading performance. To suppress this “feature noise”, we propose a novel learning method that relies on an adaptive weighted downsampling layer, a smooth-oriented convolutional block, and disturbance suppression learning. These components effectively reduce feature noise during downsampling and convolution operations, enabling the model to learn disturbance-invariant features. Furthermore, we discover that high-bit-depth RAW images can better preserve richer scene information in low-light conditions compared to typical camera sRGB outputs, thus supporting the use of RAW-input algorithms. Our analysis indicates that high bit-depth can be critical for low-light instance segmentation. To mitigate the scarcity of annotated RAW datasets, we leverage a low-light RAW synthetic pipeline to generate realistic low-light data. In addition, to facilitate further research in this direction, we capture a real-world low-light instance segmentation dataset comprising over two thousand paired low/normal-light images with instance-level pixel-wise annotations. Remarkably, without any image preprocessing, we achieve satisfactory performance on instance segmentation in very low light (4% AP higher than state-of-the-art competitors), meanwhile opening new opportunities for future research. Our code and dataset are publicly available to the community ( https://github.com/Linwei-Chen/LIS ).
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-023-01808-8