EBStereo: edge-based loss function for real-time stereo matching

Deep learning-based stereo matching has made significant progress, but it still faces challenges: The disparity prediction error maps of current models show that errors are concentrated primarily on object boundaries. We find that executing the smooth L1 loss function on the entire region during ste...

Full description

Saved in:
Bibliographic Details
Published in:The Visual computer Vol. 40; no. 4; pp. 2975 - 2986
Main Authors: Bi, Weijie, Chen, Ming, Wu, Dongliu, Lu, Shenglian
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.04.2024
Springer Nature B.V
Subjects:
Accuracy
Algorithms
Artificial Intelligence
Computer Graphics
Computer Science
Deep learning
Design
Embedding
Frames per second
Image Processing and Computer Vision
Matching
Neural networks
Original Article
Real time
Deep learning
Loss function
Feature fusion
Stereo matching
Bilateral grid learning
ISSN:0178-2789, 1432-2315
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning-based stereo matching has made significant progress, but it still faces challenges: The disparity prediction error maps of current models show that errors are concentrated primarily on object boundaries. We find that executing the smooth L1 loss function on the entire region during stereo matching model training cannot effectively address the imbalance between edge regions and flat regions, resulting in poor disparity estimates for edge regions. In this paper, a new weighted smooth L1 loss function, which considers the loss function calculation on edge regions and can yield improved accuracy, is proposed. An improved bilateral grid upsampling module is also added to the training model, and a strategy is adopted to balance the computational consumption introduced by the new loss function-weighted item, allowing for real-time inference. Extensive experiments conducted on two datasets, i.e., Scene Flow and KITTI, verify the simplicity and effectiveness of this approach. Under the condition of 33 frames per second (FPS), the endpoint error of the proposed model can be improved to 0.63. In addition, the proposed edge-based loss function can be easily embedded into many existing stereo matching networks, such as GwcNet, AANet, and PSMNet. After embedding the proposed edge-based loss function, the reduction rates of the endpoint errors of the existing models can be improved to 3.5%, 11.6%, and 27.2% for GwcNet, AANet, and PSMNet, respectively.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0178-2789
1432-2315
DOI:10.1007/s00371-023-03002-w