Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images

Recovering the 3D shape of an object from single or multiple images with deep neural networks has been attracting increasing attention in the past few years. Mainstream works (e.g. 3D-R2N2) use recurrent neural networks (RNNs) to sequentially fuse feature maps of input images. However, RNN-based app...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of computer vision Jg. 128; H. 12; S. 2919 - 2935
Hauptverfasser: Xie, Haozhe, Yao, Hongxun, Zhang, Shengping, Zhou, Shangchen, Sun, Wenxiu
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.12.2020
Springer
Schlagworte:
Artificial Intelligence
Computer Imaging
Computer Science
Image Processing and Computer Vision
Neural networks
Pattern Recognition
Pattern Recognition and Graphics
Vision
Multi-scale
Context-aware
Convolutional neural network
3D object reconstruction
ISSN:0920-5691, 1573-1405
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Recovering the 3D shape of an object from single or multiple images with deep neural networks has been attracting increasing attention in the past few years. Mainstream works (e.g. 3D-R2N2) use recurrent neural networks (RNNs) to sequentially fuse feature maps of input images. However, RNN-based approaches are unable to produce consistent reconstruction results when given the same input images with different orders. Moreover, RNNs may forget important features from early input images due to long-term memory loss. To address these issues, we propose a novel framework for single-view and multi-view 3D object reconstruction, named Pix2Vox++. By using a well-designed encoder-decoder, it generates a coarse 3D volume from each input image. A multi-scale context-aware fusion module is then introduced to adaptively select high-quality reconstructions for different parts from all coarse 3D volumes to obtain a fused 3D volume. To further correct the wrongly recovered parts in the fused 3D volume, a refiner is adopted to generate the final output. Experimental results on the ShapeNet, Pix3D, and Things3D benchmarks show that Pix2Vox++ performs favorably against state-of-the-art methods in terms of both accuracy and efficiency.
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-020-01347-6