Spherical Image Generation From a Few Normal-Field-of-View Images by Considering Scene Symmetry

Spherical images taken in all directions (360 degrees by 180 degrees) can represent an entire space including the subject, providing free direction viewing and an immersive experience to viewers. It is convenient and expands the usage scenarios to generate a spherical image from a few normal-field-o...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE Transactions on Pattern Analysis and Machine Intelligence Jg. 45; H. 5; S. 1 - 15
Hauptverfasser: Hara, Takayuki, Mukuta, Yusuke, Harada, Tatsuya
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States IEEE 01.05.2023
Institute of Electrical and Electronics Engineers (IEEE)
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Cameras
Conditional variational autoencoder
Field of view
Gravity
image generation
Image processing
Image reconstruction
Image synthesis
Recording
Rendering (computer graphics)
spherical image
Symmetry
symmetry control
symmetry estimation
Task analysis
ISSN:0162-8828, 1939-3539, 1939-3539, 2160-9292
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Spherical images taken in all directions (360 degrees by 180 degrees) can represent an entire space including the subject, providing free direction viewing and an immersive experience to viewers. It is convenient and expands the usage scenarios to generate a spherical image from a few normal-field-of-view (NFOV) images, which are partial observations. The primary challenge is generating a plausible image and controlling the high degree of freedom involved in generating a wide area that includes all directions. We focus on scene symmetry, which is a basic property of the global structure of spherical images, such as the rotational and plane symmetries. We propose a method for generating a spherical image from a few NFOV images and controlling the generated regions using scene symmetry. We incorporate the intensity of the symmetry as a latent variable into conditional variational autoencoders to estimate the possible range of symmetry and decode a spherical image whose features are represented through a combination of symmetric transformations of the NFOV image features. Our experiments show that the proposed method can generate various plausible spherical images controlled from asymmetrically to symmetrically, and can reduce the reconstruction errors of the generated images based on the estimated symmetry.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0162-8828
1939-3539
1939-3539
2160-9292
DOI:10.1109/TPAMI.2022.3215933