HeadNeRF: A Realtime NeRF-based Parametric Head Model
In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the human head. It can render high fidelity head images in real-time on modern GPUs, and supports directly controlling the generated images'...
Gespeichert in:
| Veröffentlicht in: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) S. 20342 - 20352 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
01.06.2022
|
| Schlagworte: | |
| ISSN: | 1063-6919 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the human head. It can render high fidelity head images in real-time on modern GPUs, and supports directly controlling the generated images' rendering pose and various semantic attributes. Different from existing related parametric models, we use the neural radiance fields as a novel 3D proxy instead of the traditional 3D textured mesh, which makes that HeadNeRF is able to generate high fidelity images. However, the computationally expensive rendering process of the original NeRF hinders the construction of the parametric NeRF model. To address this issue, we adopt the strategy of integrating 2D neural rendering to the rendering process of NeRF and design novel loss terms. As a result, the rendering speed of HeadNeRF can be significantly accelerated, and the rendering time of one frame is reduced from 5s to 25ms. The well designed loss terms also improve the rendering accuracy, and the fine-level details of the human head, such as the gaps between teeth, wrinkles, and beards, can be represented and synthesized by HeadNeRF. Extensive experimental results and several applications demonstrate its effectiveness. The trained parametric model is available at https://github.com/CrisHY1995/headnerf. |
|---|---|
| AbstractList | In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the human head. It can render high fidelity head images in real-time on modern GPUs, and supports directly controlling the generated images' rendering pose and various semantic attributes. Different from existing related parametric models, we use the neural radiance fields as a novel 3D proxy instead of the traditional 3D textured mesh, which makes that HeadNeRF is able to generate high fidelity images. However, the computationally expensive rendering process of the original NeRF hinders the construction of the parametric NeRF model. To address this issue, we adopt the strategy of integrating 2D neural rendering to the rendering process of NeRF and design novel loss terms. As a result, the rendering speed of HeadNeRF can be significantly accelerated, and the rendering time of one frame is reduced from 5s to 25ms. The well designed loss terms also improve the rendering accuracy, and the fine-level details of the human head, such as the gaps between teeth, wrinkles, and beards, can be represented and synthesized by HeadNeRF. Extensive experimental results and several applications demonstrate its effectiveness. The trained parametric model is available at https://github.com/CrisHY1995/headnerf. |
| Author | Liu, Ligang Xiao, Haiyao Hong, Yang Peng, Bo Zhang, Juyong |
| Author_xml | – sequence: 1 givenname: Yang surname: Hong fullname: Hong, Yang email: hymath@mail.ustc.edu.cn organization: University of Science and Technology of China – sequence: 2 givenname: Bo surname: Peng fullname: Peng, Bo email: pb15881461858@mail.ustc.edu.cn organization: University of Science and Technology of China – sequence: 3 givenname: Haiyao surname: Xiao fullname: Xiao, Haiyao email: xhy1999512@mail.ustc.edu.cn organization: University of Science and Technology of China – sequence: 4 givenname: Ligang surname: Liu fullname: Liu, Ligang email: lgliu@ustc.edu.cn organization: University of Science and Technology of China – sequence: 5 givenname: Juyong surname: Zhang fullname: Zhang, Juyong email: juyong@ustc.edu.cn organization: University of Science and Technology of China |
| BookMark | eNotjMFKw0AQQFdRsK39Aj3sDyTO7G4mu95KsFaoWoJ6LZPsBCJJK0ku_r0WPT14PN5cXRyOB1HqFiFFhHBXfOzKzJD3qQFjUsCQ2zM1R6LMUXBkz9UMgWxCAcOVWo7jJwBYg0jBz1S2EY4vUq7v9UqXwt3U9qJPIql4lKh3PHAv09DW-pTq52OU7lpdNtyNsvznQr2vH96KTbJ9fXwqVtukRfRTUjsCiD6vvM8bE9k1woDETsg0hoNU8ZeGTDTBUu1iJXXjMIbG-5ixtwt18_dtRWT_NbQ9D9_74D1YF-wPn09Gtw |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52688.2022.01973 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 1665469463 9781665469463 |
| EISSN | 1063-6919 |
| EndPage | 20352 |
| ExternalDocumentID | 9880349 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Key R&D Program of China grantid: 2020YFC1523102 funderid: 10.13039/501100012166 – fundername: Fundamental Research Funds for the Central Universities grantid: WK3470000021 funderid: 10.13039/501100012226 – fundername: National Natural Science Foundation of China grantid: 62122071,62025207 funderid: 10.13039/501100001809 – fundername: Youth Innovation Promotion Association CAS grantid: 2018495 funderid: 10.13039/501100004739 |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i118t-c4600d87b887f2da4fea016a4e62f2a9ebd2f2262d2936c4dbecf41d9f88d5a83 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 02:15:09 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i118t-c4600d87b887f2da4fea016a4e62f2a9ebd2f2262d2936c4dbecf41d9f88d5a83 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_9880349 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-June |
| PublicationDateYYYYMMDD | 2022-06-01 |
| PublicationDate_xml | – month: 06 year: 2022 text: 2022-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.6395912 |
| Snippet | In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 20342 |
| SubjectTerms | Computer vision Ethics Face and gestures; 3D from multi-view and sensors; 3D from single images; Representation learning; Vision + graphics Head Real-time systems Rendering (computer graphics) Semantics Three-dimensional displays |
| Title | HeadNeRF: A Realtime NeRF-based Parametric Head Model |
| URI | https://ieeexplore.ieee.org/document/9880349 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09a8MwED2S0KFT2ial32joWCW2LOujWwkNGUowoQ3ZgmRJEGiTko_-_uock3bo0slGCAx3st-T_N4dwH0kITkvVUrTIBLKmc2oZUzSSEaFYi4CeOVynb7I8VjNZrpowMPBC-O9r8Rnvoe31b98typ3eFTW13GxZVw3oSml2Hu1DucpWdzJCK1qd1ya6P5gWkywmAkKuBjrRS6DzdF_9VCpIGTY_t_DT6D748UjxQFlTqHhl2fQrskjqV_NTQfyUczW2E-Gj-SJTFDVtfjwBAcoIpUjhUEdFhbkJziVYBe09y68DZ9fByNa90Sgi7gV2NKSR4bilLTx4xCYMzx4E1mb4V6wwIz21sUrE8xFHBcldzFHgadOB6VcblR2Dq3laukvgBjrEu4zl-CMoLRKpRHWeatcqYLML6GDUZh_7stezOsAXP09fA3HGOa9iuoGWtv1zt_CUfm1XWzWd1WuvgE6LpNf |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH7MKehp6ib-NgePZmvTNE28iTgmzlLGHLuNtElgoJvsh3-_eV2ZHrx4agmBwntpvy_p-94HcOtJSMwLGdLQiYBylkc0ZyyhnowKyYwH8FLlOuonaSrHY5XV4G6rhbHWlsVnto235b98My_WeFTWUX6xRVztwC46Z1Vqre2JSuT3MkLJSh8XBqrzOMoG2M4ES7gYa3s2g_bov1xUShDpNv73-ENo_ajxSLbFmSOo2dkxNCr6SKqXc9mEuOfzldpB9548kAHWdU0_LMEBilhlSKaxEgtb8hOcStAH7b0Fb92n4WOPVq4IdOo3AytacM9RjExy_3lwzGjurPa8TXMrmGNa2dz4KxPMeCQXBTc-S46HRjkpTaxldAL12XxmT4Ho3ATcRibAGU4qGSZa5Mbm0hTSJfEZNDEKk89N44tJFYDzv4dvYL83fO1P-s_pywUcYMg3NVWXUF8t1vYK9oqv1XS5uC7z9g0ZrZao |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=HeadNeRF%3A+A+Realtime+NeRF-based+Parametric+Head+Model&rft.au=Hong%2C+Yang&rft.au=Peng%2C+Bo&rft.au=Xiao%2C+Haiyao&rft.au=Liu%2C+Ligang&rft.date=2022-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=20342&rft.epage=20352&rft_id=info:doi/10.1109%2FCVPR52688.2022.01973&rft.externalDocID=9880349 |