NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions
Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint suffers from occlusions, shape and texture ambiguities, motions, etc. To mitigate the problem, it is essential to build a training dataset that c...
Uloženo v:
| Vydáno v: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 8834 - 8845 |
|---|---|
| Hlavní autoři: | , , , , , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.06.2023
|
| Témata: | |
| ISSN: | 1063-6919 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint suffers from occlusions, shape and texture ambiguities, motions, etc. To mitigate the problem, it is essential to build a training dataset that captures free-viewpoint interactions. We construct a dense multi-view dome to acquire a complex human object interaction dataset, named HODome, that consists of ~71 M frames on 10 subjects interacting with 23 objects. To process the HODome dataset, we develop NeuralDome, a layer-wise neural processing pipeline tailored for multi-view video inputs to conduct accurate tracking, geometry reconstruction and free-view rendering, for both human subjects and objects. Extensive experiments on the HODome dataset demonstrate the effectiveness of NeuralDome on a variety of inference, modeling, and rendering tasks. Both the dataset and the NeuralDome tools will be disseminated to the community for further development, which can be found at https://juzezhang.github.io/NeuralDome |
|---|---|
| AbstractList | Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint suffers from occlusions, shape and texture ambiguities, motions, etc. To mitigate the problem, it is essential to build a training dataset that captures free-viewpoint interactions. We construct a dense multi-view dome to acquire a complex human object interaction dataset, named HODome, that consists of ~71 M frames on 10 subjects interacting with 23 objects. To process the HODome dataset, we develop NeuralDome, a layer-wise neural processing pipeline tailored for multi-view video inputs to conduct accurate tracking, geometry reconstruction and free-view rendering, for both human subjects and objects. Extensive experiments on the HODome dataset demonstrate the effectiveness of NeuralDome on a variety of inference, modeling, and rendering tasks. Both the dataset and the NeuralDome tools will be disseminated to the community for further development, which can be found at https://juzezhang.github.io/NeuralDome |
| Author | Yang, Hongdi Wu, Qianyang Yu, Jingyi Xu, Xinru Wang, Jingya Shi, Ye Luo, Haimin Zhang, Juze Xu, Lan |
| Author_xml | – sequence: 1 givenname: Juze surname: Zhang fullname: Zhang, Juze email: zhangjz@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 2 givenname: Haimin surname: Luo fullname: Luo, Haimin email: luohm@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 3 givenname: Hongdi surname: Yang fullname: Yang, Hongdi email: yanghd@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 4 givenname: Xinru surname: Xu fullname: Xu, Xinru email: xuxr2022@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 5 givenname: Qianyang surname: Wu fullname: Wu, Qianyang email: wuqy2022@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 6 givenname: Ye surname: Shi fullname: Shi, Ye email: shiye@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 7 givenname: Jingyi surname: Yu fullname: Yu, Jingyi email: yujingyi@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 8 givenname: Lan surname: Xu fullname: Xu, Lan email: xulan1@shanghaitech.edu.cn organization: ShanghaiTech University – sequence: 9 givenname: Jingya surname: Wang fullname: Wang, Jingya email: wangjingya@shanghaitech.edu.cn organization: ShanghaiTech University |
| BookMark | eNotjF1PwjAUQKvRRET-AQ_9A8PbXrrd-kZQhISvGOWVdN2dKRkd2UaM_14NPp1zXs69uIl1ZCGGCkZKgX2c7rZvRmfajjRoHAGQwSsxsJklNICgtKVr0VOQYpJaZe_EoG0PAIBaqdRST6zXfG5c9Vwf-UlO5KXkqi64CvFTbsPpT1jWUa7OVReSXeAvOT8fXUw2-YF9Jxex48b5LtSxfRC3pataHvyzLz5mL-_TebLcvC6mk2USNIy7xPiysNqBN8aMkYBy8ujyHHxh8rHTpJXPNOUFZoxIviSvS0tlShkWFhz2xfDyDcy8PzXh6JrvvYLfuzYWfwDr2lE6 |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52729.2023.00853 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore Digital Library (LUT) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 9798350301298 |
| EISSN | 1063-6919 |
| EndPage | 8845 |
| ExternalDocumentID | 10204259 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i204t-5cfd92a0c55543808b8c3abb0cd5b4a2821c728bd37e338cf8c2f98f6873d90a3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 16 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001062522101013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:56:29 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-5cfd92a0c55543808b8c3abb0cd5b4a2821c728bd37e338cf8c2f98f6873d90a3 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_10204259 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-June |
| PublicationDateYYYYMMDD | 2023-06-01 |
| PublicationDate_xml | – month: 06 year: 2023 text: 2023-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.4464157 |
| Snippet | Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 8834 |
| SubjectTerms | 3D from multi-view and sensors Computer vision Geometry Pipelines Rendering (computer graphics) Shape Training Visualization |
| Title | NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions |
| URI | https://ieeexplore.ieee.org/document/10204259 |
| WOSCitedRecordID | wos001062522101013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV25TgMxELVIREEVjiBuuaB1cLyHbToUEVGFFYIoXbS2x9IWbKIc8Pt4vEugoaCz3ezKh_zmed48Qm4DAlAccs-kEZal4DgzkBuWpK4ELz0-DkWzCTmZqNlMF61YPWphACAmn8EAm_Et3y3sFqmycMIF7jHdIR0p80astSNUkhDK5Fq18rgh13ejafGSiYAeB-gRjoVM0QL5l4lKvEPGvX9-_ZD0f9R4tNjdM0dkD-pj0mvhI20P5_qETLDQBua-vcM9faBNj6LZGUrOaVEtsQF0UdMou2XTCj5pZPHZs0E-hkZ-sJE6rPvkbfz4OnpirV0Cq8KPbVhmvdOi5DYLECFRXBllk9IYbl1m0jLEVkMrhTIukRACU-uVFV4rnyuZOM3L5JR060UNZ4SCDyhAaQARoinltRkq452TIZo0INP8nPRxfubLpiLG_HtqLv4YvyQHuARNitUV6W5WW7gm-_ZjU61XN3EdvwAwtp6F |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LTgMhFCVaTXRVHzW-ZeGWSpkH4M40mhrrODG16a4Z4JLMwrbpQ39fYMbqxoU7YDMTHuHcwz33IHTtEICgkFrCFdMkBkOJglSRKDYFWG7941Awm-BZJkYjmddi9aCFAYCQfAZt3wxv-WaqV54qcyec-T0mN9FWEseMVnKtNaUSuWAmlaIWyHWovOkO89eEOfzY9i7hvpSpN0H-ZaMSbpGH5j-_v4daP3o8nK9vmn20AZMD1KwBJK6P5-IQZb7Uhs9-e4dbfIerHvZ2Z150jvNy5huApxMchLdkWMInDjw-eVGekcGBIazEDosWenu4H3R7pDZMIKX7sSVJtDWSFVQnDiREggoldFQoRbVJVFy46KqjORPKRBxcaKqt0MxKYVPBIyNpER2hxmQ6gWOEwTocICQAc_GUsFJ1hLLGcBdPKuBxeoJafn7Gs6omxvh7ak7_GL9CO73Bc3_cf8yeztCuX44q4eocNZbzFVygbf2xLBfzy7CmX6Bqocw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=NeuralDome%3A+A+Neural+Modeling+Pipeline+on+Multi-View+Human-Object+Interactions&rft.au=Zhang%2C+Juze&rft.au=Luo%2C+Haimin&rft.au=Yang%2C+Hongdi&rft.au=Xu%2C+Xinru&rft.date=2023-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=8834&rft.epage=8845&rft_id=info:doi/10.1109%2FCVPR52729.2023.00853&rft.externalDocID=10204259 |