Deep Stereo Using Adaptive Thin Volume Representation With Uncertainty Awareness
We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geometry from multi-view images. Previous learning-based MVS methods estimate per-view depth using plane sweep volumes (PSVs)...
Uloženo v:
| Vydáno v: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 2521 - 2531 |
|---|---|
| Hlavní autoři: | , , , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.06.2020
|
| Témata: | |
| ISSN: | 1063-6919 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geometry from multi-view images. Previous learning-based MVS methods estimate per-view depth using plane sweep volumes (PSVs) with a fixed depth hypothesis at each plane; this requires densely sampled planes for high accuracy, which is impractical for high-resolution depth because of limited memory. In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to the uncertainties of previous per-pixel depth predictions. Our UCS-Net has three stages: the first stage processes a small PSV to predict low-resolution depth; two ATVs are then used in the following stages to refine the depth with higher resolution and higher accuracy. Our ATV consists of only a small number of planes with low memory and computation costs; yet, it efficiently partitions local depth ranges within learned small uncertainty intervals. We propose to use variance-based uncertainty estimates to adaptively construct ATVs; this differentiable process leads to reasonable and fine-grained spatial partitioning. Our multi-stage framework progressively sub-divides the vast scene space with increasing depth resolution and precision, which enables reconstruction with high completeness and accuracy in a coarse-to-fine fashion. We demonstrate that our method achieves superior performance compared with other learning-based MVS methods on various challenging datasets. |
|---|---|
| AbstractList | We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geometry from multi-view images. Previous learning-based MVS methods estimate per-view depth using plane sweep volumes (PSVs) with a fixed depth hypothesis at each plane; this requires densely sampled planes for high accuracy, which is impractical for high-resolution depth because of limited memory. In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to the uncertainties of previous per-pixel depth predictions. Our UCS-Net has three stages: the first stage processes a small PSV to predict low-resolution depth; two ATVs are then used in the following stages to refine the depth with higher resolution and higher accuracy. Our ATV consists of only a small number of planes with low memory and computation costs; yet, it efficiently partitions local depth ranges within learned small uncertainty intervals. We propose to use variance-based uncertainty estimates to adaptively construct ATVs; this differentiable process leads to reasonable and fine-grained spatial partitioning. Our multi-stage framework progressively sub-divides the vast scene space with increasing depth resolution and precision, which enables reconstruction with high completeness and accuracy in a coarse-to-fine fashion. We demonstrate that our method achieves superior performance compared with other learning-based MVS methods on various challenging datasets. |
| Author | Xu, Zexiang Zhu, Shilin Li, Zhuwen Su, Hao Ramamoorthi, Ravi Li, Li Erran Cheng, Shuo |
| Author_xml | – sequence: 1 givenname: Shuo surname: Cheng fullname: Cheng, Shuo organization: University of California, San Diego – sequence: 2 givenname: Zexiang surname: Xu fullname: Xu, Zexiang organization: University of California, San Diego – sequence: 3 givenname: Shilin surname: Zhu fullname: Zhu, Shilin organization: University of California, San Diego – sequence: 4 givenname: Zhuwen surname: Li fullname: Li, Zhuwen organization: Nuro Inc – sequence: 5 givenname: Li Erran surname: Li fullname: Li, Li Erran organization: Scale AI; Columbia University – sequence: 6 givenname: Ravi surname: Ramamoorthi fullname: Ramamoorthi, Ravi organization: University of California, San Diego – sequence: 7 givenname: Hao surname: Su fullname: Su, Hao organization: University of California, San Diego |
| BookMark | eNotjNFOwjAUQKvRRES-QB_6A8Pb3rVrHwmKmpBIcOAjGdut1EC3rFXD30uiDyfnPJ1rdhHaQIzdCRgLAfZ-ul4sc6kBxhIkjAFOfcZGtjCikCeENuqcDQRozLQV9oqNYvwEAJRCaGsGbPFA1PG3RD21fBV9-OCTpuqS_yZe7nzg63b_dSC-pK6nSCFVybeBv_u046tQU58qH9KRT36qngLFeMMuXbWPNPr3kJWzx3L6nM1fn16mk3nmJWDKnFOFgcblEpotOcqN0Y2pkQgBwRpqsMa6diC22jpUW4UFKiCjcmWcxSG7_dt6Itp0vT9U_XFjhdKYS_wFnwdSWA |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR42600.2020.00260 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 9781728171685 1728171687 |
| EISSN | 1063-6919 |
| EndPage | 2531 |
| ExternalDocumentID | 9156342 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i203t-ff5780df420dbefe4886d8c3ee303098ed3c3ccf01b69f35b537350e85458f93 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 290 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000620679502079&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:30:35 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-ff5780df420dbefe4886d8c3ee303098ed3c3ccf01b69f35b537350e85458f93 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_9156342 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-Jun |
| PublicationDateYYYYMMDD | 2020-06-01 |
| PublicationDate_xml | – month: 06 year: 2020 text: 2020-Jun |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2020 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.6122873 |
| Snippet | We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 2521 |
| SubjectTerms | Feature extraction Geometry Image reconstruction Image resolution Memory management Three-dimensional displays Uncertainty |
| Title | Deep Stereo Using Adaptive Thin Volume Representation With Uncertainty Awareness |
| URI | https://ieeexplore.ieee.org/document/9156342 |
| WOSCitedRecordID | wos000620679502079&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZKxcBUoEW85YGRUCeOX2NVqJiqqJTSrYqds-iSVm0K4t9jO1FhYGGzPNjSne7su_u-O4TuBLHMxImOmAIXoBhniopyG2mWSy05zzW1YdiEGI_lfK6yFrrfc2EAIIDP4MEvQy2_WJmdT5X1lQs2aOoc7oEQvOZq7fMp1EUyXMmGHRcT1R_Osknov-6iwMQDuBLfh_LXDJXwhIw6_7v8GPV-uHg4278yJ6gF5SnqNJ9H3JjmtouyR4A1fnFighUOQAA8KPK192bYD-fEs-CH8CRAXxvGUYnfltU7fnVHBGRA9YUHn54e5vxfD01HT9Phc9SMS4iWCaFVZK2zPlLYNCGFBgvONHkhDQWgvo4ioaCGGmNJrLmylGlGBWUEpK-dWUXPULtclXCOMNPALLEcFFPuu5Vq4JakwuQxyEJAfoG6Xj6Ldd0QY9GI5vLv7St05BVQ46uuUbva7OAGHZqParnd3AYtfgMAXp8N |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTwIxEG0ImugJFYzf9uDRlbLddtsjQQlGJBtE5Ea23WnkshBYNP5727JBD168NT20yTQz7XTem4fQTUwM061QBUyCTVC0dUVJuQkUS4USnKeKGi82EQ8GYjKRSQXdbrkwAODBZ3Dnhr6Wn8312n2VNaVNNmhkA-6OU84q2VrbHxVqcxkuRcmPaxHZ7IyToe_AbvPA0EG4QteJ8peKir9EurX_bX-AGj9sPJxs75lDVIH8CNXK5yMunXNVR8k9wAK_WEPBHHsoAG5n6cLFM-zkOfHYRyI89ODXknOU47dZ8Y5f7RIeG1B84fanI4jZCNhAo-7DqNMLSsGEYBYSWgTGWP8jmYlCkikwYJ2TZ0JTAOoqKQIyqqnWhrQUl4YyxWhMGQHhqmdG0mNUzec5nCDMFDBDDAfJpH1wRQq4IVGs0xaILIb0FNWdfaaLTUuMaWmas7-nr9Feb_Tcn_YfB0_naN8dxgZtdYGqxXINl2hXfxSz1fLKn-g3eHuiVg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Deep+Stereo+Using+Adaptive+Thin+Volume+Representation+With+Uncertainty+Awareness&rft.au=Cheng%2C+Shuo&rft.au=Xu%2C+Zexiang&rft.au=Zhu%2C+Shilin&rft.au=Li%2C+Zhuwen&rft.date=2020-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=2521&rft.epage=2531&rft_id=info:doi/10.1109%2FCVPR42600.2020.00260&rft.externalDocID=9156342 |