Stylizing 3D Scene via Implicit Representation and HyperNetwork

In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches, which often leads to blurry results or inconsis...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / IEEE Workshop on Applications of Computer Vision s. 215 - 224
Hlavní autoři: Chiang, Pei-Ze, Tsai, Meng-Shiun, Tseng, Hung-Yu, Lai, Wei-Sheng, Chiu, Wei-Chen
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.01.2022
Témata:
3D Computer Vision Neural rendering
Computer vision
Optimization
Prediction algorithms
Predictive models
Solid modeling
Three-dimensional displays
Training
ISSN:2642-9381
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches, which often leads to blurry results or inconsistent appearance. Inspired by the high-quality results of the neural radiance fields (NeRF) method, we propose a joint framework to directly render novel views with the desired style. Our framework consists of two components: an implicit representation of the 3D scene with the neural radiance fields model, and a hypernetwork to transfer the style information into the scene representation. To alleviate the training difficulties and memory burden, we propose a two-stage training procedure and a patch sub-sampling approach to optimize the style and content losses with the neural radiance fields model. After optimization, our model is able to render consistent novel views at arbitrary view angles with arbitrary style. Both quantitative evaluation and human subject study have demonstrated that the proposed method generates faithful stylization results with consistent appearance across different views.
ISSN:2642-9381
DOI:10.1109/WACV51458.2022.00029