Stylizing 3D Scene via Implicit Representation and HyperNetwork
In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches, which often leads to blurry results or inconsis...
Saved in:
| Published in: | Proceedings / IEEE Workshop on Applications of Computer Vision pp. 215 - 224 |
|---|---|
| Main Authors: | , , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.01.2022
|
| Subjects: | |
| ISSN: | 2642-9381 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches, which often leads to blurry results or inconsistent appearance. Inspired by the high-quality results of the neural radiance fields (NeRF) method, we propose a joint framework to directly render novel views with the desired style. Our framework consists of two components: an implicit representation of the 3D scene with the neural radiance fields model, and a hypernetwork to transfer the style information into the scene representation. To alleviate the training difficulties and memory burden, we propose a two-stage training procedure and a patch sub-sampling approach to optimize the style and content losses with the neural radiance fields model. After optimization, our model is able to render consistent novel views at arbitrary view angles with arbitrary style. Both quantitative evaluation and human subject study have demonstrated that the proposed method generates faithful stylization results with consistent appearance across different views. |
|---|---|
| AbstractList | In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches, which often leads to blurry results or inconsistent appearance. Inspired by the high-quality results of the neural radiance fields (NeRF) method, we propose a joint framework to directly render novel views with the desired style. Our framework consists of two components: an implicit representation of the 3D scene with the neural radiance fields model, and a hypernetwork to transfer the style information into the scene representation. To alleviate the training difficulties and memory burden, we propose a two-stage training procedure and a patch sub-sampling approach to optimize the style and content losses with the neural radiance fields model. After optimization, our model is able to render consistent novel views at arbitrary view angles with arbitrary style. Both quantitative evaluation and human subject study have demonstrated that the proposed method generates faithful stylization results with consistent appearance across different views. |
| Author | Tsai, Meng-Shiun Lai, Wei-Sheng Chiu, Wei-Chen Chiang, Pei-Ze Tseng, Hung-Yu |
| Author_xml | – sequence: 1 givenname: Pei-Ze surname: Chiang fullname: Chiang, Pei-Ze organization: National Yang Ming Chiao Tung University,Taiwan – sequence: 2 givenname: Meng-Shiun surname: Tsai fullname: Tsai, Meng-Shiun organization: National Yang Ming Chiao Tung University,Taiwan – sequence: 3 givenname: Hung-Yu surname: Tseng fullname: Tseng, Hung-Yu organization: University of California,Merced – sequence: 4 givenname: Wei-Sheng surname: Lai fullname: Lai, Wei-Sheng organization: University of California,Merced – sequence: 5 givenname: Wei-Chen surname: Chiu fullname: Chiu, Wei-Chen organization: National Yang Ming Chiao Tung University,Taiwan |
| BookMark | eNotjl1LwzAUQKMouE1_gT7kD3Tee7MkzZOMOd1gKDg_Hkea3kp0S0tblPnrHejTgfNwOENxkurEQlwhjBHBXb9NZ68aJzofExCNAYDckRiiMXoCDjUciwGZCWVO5Xgmhl33AaAcOjUQN-t-v40_Mb1LdSvXgRPLr-jlctdsY4i9fOKm5Y5T7_tYJ-lTKRf7htsH7r_r9vNcnFZ-2_HFP0fi5W7-PFtkq8f75Wy6yiKB6rPApig0ayATrCVboYGD0h4pt-GwXRKHErGybMhYdrbyqDi3ZSiLwgQ1Epd_3cjMm6aNO9_uN86CBUPqF0YBSzI |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/WACV51458.2022.00029 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 1665409150 9781665409155 |
| EISSN | 2642-9381 |
| EndPage | 224 |
| ExternalDocumentID | 9707062 |
| Genre | orig-research |
| GroupedDBID | 29G 29O 6IE 6IF 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL RNS |
| ID | FETCH-LOGICAL-i203t-ce6bb5e5026c7727f160e6b5a1287c202d2ecd11f7e6267e97fa13e87dcdbb6c3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 74 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000800471200022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:49:39 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-ce6bb5e5026c7727f160e6b5a1287c202d2ecd11f7e6267e97fa13e87dcdbb6c3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_9707062 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-Jan. |
| PublicationDateYYYYMMDD | 2022-01-01 |
| PublicationDate_xml | – month: 01 year: 2022 text: 2022-Jan. |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings / IEEE Workshop on Applications of Computer Vision |
| PublicationTitleAbbrev | WACV |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0039193 |
| Score | 2.559946 |
| Snippet | In this work, we aim to address the 3D scene stylization problem - generating stylized images of the scene at arbitrary novel view angles. A straightforward... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 215 |
| SubjectTerms | 3D Computer Vision Neural rendering Computer vision Optimization Prediction algorithms Predictive models Solid modeling Three-dimensional displays Training |
| Title | Stylizing 3D Scene via Implicit Representation and HyperNetwork |
| URI | https://ieeexplore.ieee.org/document/9707062 |
| WOSCitedRecordID | wos000800471200022&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zePA0dRN_k4NH45Z2aZqjTMcEGUN07jbS5AUK0snWDfSv9yWt04MXb-VRKH1N3_vey_e9EHIVJ85kOhUM05NgfccVU9YkzDcbnN_2cUGVNn2U43E6m6lJg1xvtTAAEMhncOMvw16-XZi1b5V1lcQF6gPujpSy0mp9R91YIRKppXG8p7qvt4MpYgHh2VtRmMnpQeSvA1RC_hi2_vfkfdL5EeLRyTbFHJAGFIekVSNHWv-XqzZC7vLjLf_Ee2h8h3YMYHSTa_oQ-OJ5SZ8C4bXWGRVUF5aOsAJdjisWeIe8DO-fByNWH43A8qgXl8xAkmUCBFZQBvGxdDzpoUloTDfS4CvbCIzl3EnAikWCkk7zGFJpjc2yxMRHpFksCjgmVOk-Ty0IcFiZpTLSKoHUj2HrW0RTEJ-QtvfH_L2afjGvXXH6t_mM7HmHV02Kc9Isl2u4ILtmU-ar5WX4ZF8QJpcq |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fS8MwEA5jCvo0dRN_mwcfrVvapmkeZSobzjFkzr2NNrlAQTrZuoH-9V7SOn3wxbdyFEqv6d13l--7EHIVREalScw9TE_cCw2TntQq8myzwdhtH-NUaZOBGA7j6VSOauR6o4UBAEc-gxt76fby9VytbKusLQUuUBtwt3gY-qxUa33H3UAiFqnEcawj26-33QmiAW75W76bymlh5K8jVFwGeWj879l7pPUjxaOjTZLZJzXID0ijwo60-jOXTQTdxcdb9on30OAO7RjC6DpLaN8xxrOCPjvKa6U0ymmSa9rDGnQxLHngLfLycD_u9rzqcAQv8ztB4SmI0pQDxxpKIUIWhkUdNPEEE45Q-MraB6UZMwKwZhEghUlYALHQSqdppIJDUs_nORwRKpOQxRo4GKzNYuEnMoLYDmILNeIpCI5J0_pj9l7Ov5hVrjj523xJdnrjp8Fs0B8-npJd6_yyZXFG6sViBedkW62LbLm4cJ_vCx4lmnE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%2F+IEEE+Workshop+on+Applications+of+Computer+Vision&rft.atitle=Stylizing+3D+Scene+via+Implicit+Representation+and+HyperNetwork&rft.au=Chiang%2C+Pei-Ze&rft.au=Tsai%2C+Meng-Shiun&rft.au=Tseng%2C+Hung-Yu&rft.au=Lai%2C+Wei-Sheng&rft.date=2022-01-01&rft.pub=IEEE&rft.eissn=2642-9381&rft.spage=215&rft.epage=224&rft_id=info:doi/10.1109%2FWACV51458.2022.00029&rft.externalDocID=9707062 |