In EDS ansehen

Unpaired semantic neural person image synthesis.

Gespeichert in:
Bibliographische Detailangaben
Titel: Unpaired semantic neural person image synthesis.
Autoren: Liu, Yixiu, Jiang, Tao, Si, Pengju, Zhu, Shangdong, Yan, Chenggang, Wang, Shuai, Yin, Haibing
Quelle: Visual Computer; Jan2025, Vol. 41 Issue 1, p383-397, 15p
Schlagwörter: GENERATIVE adversarial networks, COGNITIVE psychology, ARTIFICIAL intelligence, INFORMATION networks, IMAGE processing
Abstract: Pose-guided person image synthesis is a challenging task that aims to generate photo-realistic images of a person with the same appearance as a source image but the same pose as a target image. Existing methods often suffer from noticeable artifacts due to the omission of multi-view information, and the requirement for paired source–target images in certain methods during training further limits the application of the models. To address these issues, we present a semantic neural person image synthesis framework, named SNPIS, which leverages neural radiance fields (NeRF) to synthesize high-fidelity human images of arbitrary poses from multi-view source images and target semantic maps. First, we introduce the semantic mirror orientation adjustment that forces sampling points to focus on the human body, effectively suppressing background interference and enhancing human details. Then we devise a NeRF-based appearance-shape decoupling generative adversarial network, which separates appearance and shape of a shared volume generated from multi-view source images and corresponding semantic maps. Finally, we use the obtained decoupled generator to synthesize human images guided by the target semantic maps, employing appearance inversion, and optimizing pose reconstruction with semantic consistency constraint. Experimental results show that our approach not only outperforms existing unpaired pose-guided person image synthesis methods, but also competes with many paired methods. [ABSTRACT FROM AUTHOR]
Copyright of Visual Computer is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Datenbank: Complementary Index
Beschreibung
Abstract:Pose-guided person image synthesis is a challenging task that aims to generate photo-realistic images of a person with the same appearance as a source image but the same pose as a target image. Existing methods often suffer from noticeable artifacts due to the omission of multi-view information, and the requirement for paired source–target images in certain methods during training further limits the application of the models. To address these issues, we present a semantic neural person image synthesis framework, named SNPIS, which leverages neural radiance fields (NeRF) to synthesize high-fidelity human images of arbitrary poses from multi-view source images and target semantic maps. First, we introduce the semantic mirror orientation adjustment that forces sampling points to focus on the human body, effectively suppressing background interference and enhancing human details. Then we devise a NeRF-based appearance-shape decoupling generative adversarial network, which separates appearance and shape of a shared volume generated from multi-view source images and corresponding semantic maps. Finally, we use the obtained decoupled generator to synthesize human images guided by the target semantic maps, employing appearance inversion, and optimizing pose reconstruction with semantic consistency constraint. Experimental results show that our approach not only outperforms existing unpaired pose-guided person image synthesis methods, but also competes with many paired methods. [ABSTRACT FROM AUTHOR]
ISSN:01782789
DOI:10.1007/s00371-024-03331-4