Simultaneous estimation of projector and camera poses for multiple oneshot scan using pixel-wise correspondences estimated by U-Nets and GCN

Dense and accurate 3D shape acquisition of objects by active-stereo technique has been an important research topic and intensively researched. One of the promising fields for active-stereo techniques is medical applications, such as 3D endoscope systems. In such systems, since a sensor is dynamicall...

Full description

Saved in:
Bibliographic Details
Published in:Computer methods in biomechanics and biomedical engineering. Vol. 10; no. 5; pp. 540 - 548
Main Authors: Furukawa, Ryo, Mikamo, Michihiro, Kawasaki, Hiroshi, Sagawa, Ryusuke, Oka, Shiro, Kotachi, Takahiro, Okamoto, Yuki, Tanaka, Shinji
Format: Journal Article
Language:English
Published: Taylor & Francis 03.09.2022
Subjects:
3D endoscope
active stereo
GCN
ISSN:2168-1163, 2168-1171
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Dense and accurate 3D shape acquisition of objects by active-stereo technique has been an important research topic and intensively researched. One of the promising fields for active-stereo techniques is medical applications, such as 3D endoscope systems. In such systems, since a sensor is dynamically moved during the operation, single-frame shape reconstruction, a.k.a. oneshot scan, is necessary. For oneshot scan, there are several open problems, such as low resolution because of spatial coding, and unstable correspondence estimation between the detected patterns and the projected pattern because of irregular reflection. In this paper, we propose a solution for those problems. To increase the resolution, an accurate and stable interpolation method based on deep neural networks (DNNs) is proposed. Since most patterns used for oneshot scan are periodic, pixel-wise phase estimation can be achieved by detecting repetition in the pattern. A graph convolutional network (GCN), which is a deep neural network for graphs, is used for the correspondence problem. In the experiment, pixel-wise shape reconstruction results, as well as robust correspondence estimation using DNNs and a GCN, are shown. In addition, the effectiveness of the techniques is confirmed by comparing the proposed method with existing methods.
ISSN:2168-1163
2168-1171
DOI:10.1080/21681163.2021.2009376