Uni MS-PS: A multi-scale encoder-decoder transformer for universal photometric stereo

Photometric Stereo (PS) addresses the challenge of reconstructing a three-dimensional (3D) representation of an object by estimating the 3D normals at all points on the object’s surface. This is achieved through the analysis of at least three photographs, all taken from the same viewpoint but with d...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Computer vision and image understanding Ročník 248; s. 104093
Hlavní autoři: Hardy, Clément, Quéau, Yvain, Tschumperlé, David
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Inc 01.11.2024
Elsevier
Témata:
3D-reconstruction
Computer Science
Computer Vision and Pattern Recognition
Dataset
Multi-scale architecture
Normal map estimation
Photometric stereo
Multi-scale architecture
3D-reconstruction
Photometric stereo
Dataset
Normal map estimation
ISSN:1077-3142, 1090-235X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Photometric Stereo (PS) addresses the challenge of reconstructing a three-dimensional (3D) representation of an object by estimating the 3D normals at all points on the object’s surface. This is achieved through the analysis of at least three photographs, all taken from the same viewpoint but with distinct lighting conditions. This paper introduces a novel approach for Universal PS, i.e., when both the active lighting conditions and the ambient illumination are unknown. Our method employs a multi-scale encoder–decoder architecture based on Transformers that allows to accommodates images of any resolutions as well as varying number of input images. We are able to scale up to very high resolution images like 6000 pixels by 8000 pixels without losing performance and maintaining a decent memory footprint. Moreover, experiments on publicly available datasets establish that our proposed architecture improves the accuracy of the estimated normal field by a significant factor compared to state-of-the-art methods. Code and dataset available at: https://clement-hardy.github.io/Uni-MS-PS/index.html. •Universal photometric stereo for all lighting and environments.•3D reconstruction method that scales up to very high-resolution images.•New diverse training synthetic dataset for photometric stereo.
ISSN:1077-3142
1090-235X
DOI:10.1016/j.cviu.2024.104093