Improving neural implicit surfaces geometry with patch warping
Neural implicit surfaces have become an important technique for multi-view 3D reconstruction but their accuracy remains limited. In this paper, we argue that this comes from the difficulty to learn and render high frequency textures with neural networks. We thus propose to add to the standard neural...
Uloženo v:
| Vydáno v: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 6250 - 6259 |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.06.2022
|
| Témata: | |
| ISSN: | 1063-6919 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Neural implicit surfaces have become an important technique for multi-view 3D reconstruction but their accuracy remains limited. In this paper, we argue that this comes from the difficulty to learn and render high frequency textures with neural networks. We thus propose to add to the standard neural rendering optimization a direct photo-consistency term across the different views. Intuitively, we optimize the implicit geometry so that it warps views on each other in a consistent way. We demonstrate that two elements are key to the success of such an approach: (i) warping entire patches, using the predicted occupancy and normals of the 3D points along each ray, and measuring their similarity with a robust structural similarity (SSIM); (ii) handling visibility and occlusion in such a way that incorrect warps are not given too much importance while encouraging a reconstruction as complete as possible. We evaluate our approach, dubbed NeuralWarp, on the standard DTU and EPFL benchmarks and show it outperforms state of the art unsupervised implicit surfaces reconstructions by over 20% on both datasets. Our code is available at https://github.com/fdarmon/NeuralWarp |
|---|---|
| AbstractList | Neural implicit surfaces have become an important technique for multi-view 3D reconstruction but their accuracy remains limited. In this paper, we argue that this comes from the difficulty to learn and render high frequency textures with neural networks. We thus propose to add to the standard neural rendering optimization a direct photo-consistency term across the different views. Intuitively, we optimize the implicit geometry so that it warps views on each other in a consistent way. We demonstrate that two elements are key to the success of such an approach: (i) warping entire patches, using the predicted occupancy and normals of the 3D points along each ray, and measuring their similarity with a robust structural similarity (SSIM); (ii) handling visibility and occlusion in such a way that incorrect warps are not given too much importance while encouraging a reconstruction as complete as possible. We evaluate our approach, dubbed NeuralWarp, on the standard DTU and EPFL benchmarks and show it outperforms state of the art unsupervised implicit surfaces reconstructions by over 20% on both datasets. Our code is available at https://github.com/fdarmon/NeuralWarp |
| Author | Devaux, Jean-Clement Monasse, Pascal Aubry, Mathieu Bascle, Benedicte Darmon, Francois |
| Author_xml | – sequence: 1 givenname: Francois surname: Darmon fullname: Darmon, Francois organization: Thales LAS France – sequence: 2 givenname: Benedicte surname: Bascle fullname: Bascle, Benedicte organization: Thales LAS France – sequence: 3 givenname: Jean-Clement surname: Devaux fullname: Devaux, Jean-Clement organization: Thales LAS France – sequence: 4 givenname: Pascal surname: Monasse fullname: Monasse, Pascal organization: Univ. Gustave Eiffel, CNRS,LIGM (UMR 8049), École des Ponts,Marne-la-Vallée,France – sequence: 5 givenname: Mathieu surname: Aubry fullname: Aubry, Mathieu organization: Univ. Gustave Eiffel, CNRS,LIGM (UMR 8049), École des Ponts,Marne-la-Vallée,France |
| BookMark | eNotjM1KAzEURqMo2NY-gS7yAlPvTSZpshFksFooKKJuS5Imncj8kZla-vaO6AeHszrflFw0beMJuUVYIIK-Kz5f3wSTSi0YMLYAkCjPyBSlFLnUueTnZIIgeSY16isy7_svAOAMUWo1Iffrukvtd2z2tPGHZCoa666KLg60P6RgnO_p3re1H9KJHuNQ0s4MrqRHk7oxuiaXwVS9n_97Rj5Wj-_Fc7Z5eVoXD5usZIoPGQLYPNfCCsdwyYPTSzbCtVLgAjKrAliO3HEmduCE1TthmGd5kFY5j3xGbv5-o_d-26VYm3Ta_uZ83A8RGEt8 |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52688.2022.00616 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 1665469463 9781665469463 |
| EISSN | 1063-6919 |
| EndPage | 6259 |
| ExternalDocumentID | 9880333 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-h283t-100b4495b5c2173fc972c9739880cf12b8f0b313c325d0c5b9d5a2e24f6b8ce13 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 97 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000867754206051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:15:09 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-h283t-100b4495b5c2173fc972c9739880cf12b8f0b313c325d0c5b9d5a2e24f6b8ce13 |
| OpenAccessLink | https://hal.science/hal-04315790 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_9880333 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-June |
| PublicationDateYYYYMMDD | 2022-06-01 |
| PublicationDate_xml | – month: 06 year: 2022 text: 2022-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.595546 |
| Snippet | Neural implicit surfaces have become an important technique for multi-view 3D reconstruction but their accuracy remains limited. In this paper, we argue that... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 6250 |
| SubjectTerms | 3D from multi-view and sensors; Vision + graphics Geometry Neural networks Pattern recognition Rendering (computer graphics) Surface reconstruction Surface texture Three-dimensional displays |
| Title | Improving neural implicit surfaces geometry with patch warping |
| URI | https://ieeexplore.ieee.org/document/9880333 |
| WOSCitedRecordID | wos000867754206051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwED21FQNTgRbxLQ-MhCa-JLYXloqKqaoQoG5V7JxpJWirNAXx77HTKDCwMFiyvNg62-e783t3ANcKVRJmOQYalXEOitsL7XRlkAlpiazKLdmq2IQYj-V0qiYtuGm4MERUgc_o1nerv_x8ZbY-VDZQ7rAhYhvaQqQ7rlYTT0HnyaRK1uy4KFSD4cvk0Scz8QAu7tNypr6o-a8aKtUTMur-b_ID6P9w8dikeWUOoUXLI-jWxiOrr-amB3dNeID5HJXZG1tUYPFFyTbbwnrkFXul1TuVxRfz0Ve2dlp4zj6zwlOm-vA8un8aPgR1cYRg7iyC0qnPUMfOu9GJcV4FWqMEdw39Mo2NuJY21BihQZ7koUm0ypOME49tqqWhCI-hs1wt6QQYGSUV5UqmKOIwS5WOUUhypiPHLI_lKfS8OGbrXf6LWS2Js7-Hz2Hfy3sHp7qATlls6RL2zEe52BRX1aZ9AxKhmIA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED6VggRTgRbxxgMjoUkuD3thqUBFlKpCBXWrYudMK0FbpSmIf4-dRoGBhcGS5cXW2T7fnb_vDuBSoAjdJEVHolDGQTF7IY2udJKYayItUk26KDYR9_t8NBKDGlxVXBgiKsBndG27xV9-OlcrGyprC3PYEHEDNsMg8N01W6uKqKDxZSLBS36c54p252XwZNOZWAiXbxNzRras-a8qKsUjctf43_S70Pph47FB9c7sQY1m-9AozUdWXs5lE26qAAGzWSqTNzYt4OLTnC1XmbbYK_ZK83fKsy9m469sYfTwhH0mmSVNteD57nbY6TpleQRnYmyC3ChQVwbGv5GhMn4FaiVi3zS0y1Ta8yXXrkQPFfph6qpQijRMfPIDHUmuyMMDqM_mMzoERkpwQangEcaBm0RCBhhzMsajj0ka8CNoWnGMF-sMGONSEsd_D1_Adnf42Bv37vsPJ7BjZb8GV51CPc9WdAZb6iOfLrPzYgO_ATMGm8c |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Improving+neural+implicit+surfaces+geometry+with+patch+warping&rft.au=Darmon%2C+Francois&rft.au=Bascle%2C+Benedicte&rft.au=Devaux%2C+Jean-Clement&rft.au=Monasse%2C+Pascal&rft.date=2022-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=6250&rft.epage=6259&rft_id=info:doi/10.1109%2FCVPR52688.2022.00616&rft.externalDocID=9880333 |