Pyramid-VAE-GAN: Transferring hierarchical latent variables for image inpainting

Significant progress has been made in image inpainting methods in recent years. However, they are incapable of producing inpainting results with reasonable structures, rich detail, and sharpness at the same time. In this paper, we propose the Pyramid-VAE-GAN network for image inpainting to address t...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Computational visual media (Beijing) Ročník 9; číslo 4; s. 827 - 841
Hlavní autoři: Tian, Huiyuan, Zhang, Li, Li, Shijian, Yao, Min, Pan, Gang
Médium: Journal Article
Jazyk:angličtina
Vydáno: Beijing Tsinghua University Press 01.12.2023
Springer Nature B.V
SpringerOpen
Témata:
Artificial Intelligence
Computer Graphics
Computer Science
generative model
image inpainting
Image Processing and Computer Vision
Incompatibility
latent variable transfer (LTN)
pyramid structure
Research Article
Sharpness
User Interfaces and Human Computer Interaction
Variables
variational autoencoder (VAE)
image inpainting
latent variable transfer (LTN)
pyramid structure
generative model
variational autoencoder (VAE)
ISSN:2096-0433, 2096-0662
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Significant progress has been made in image inpainting methods in recent years. However, they are incapable of producing inpainting results with reasonable structures, rich detail, and sharpness at the same time. In this paper, we propose the Pyramid-VAE-GAN network for image inpainting to address this limitation. Our network is built on a variational autoencoder (VAE) backbone that encodes high-level latent variables to represent complicated high-dimensional prior distributions of images. The prior assists in reconstructing reasonable structures when inpainting. We also adopt a pyramid structure in our model to maintain rich detail in low-level latent variables. To avoid the usual incompatibility of requiring both reasonable structures and rich detail, we propose a novel cross-layer latent variable transfer module. This transfers information about long-range structures contained in high-level latent variables to low-level latent variables representing more detailed information. We further use adversarial training to select the most reasonable results and to improve the sharpness of the images. Extensive experimental results on multiple datasets demonstrate the superiority of our method. Our code is available at https://github.com/thy960112/Pyramid-VAE-GAN .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2096-0433
2096-0662
DOI:10.1007/s41095-022-0331-3