SPIDER: A framework for processing, editing and presenting immersive high-resolution spherical indoor scenes
Today’s Extended Reality (XR) applications that call for specific Diminished Reality (DR) strategies to hide specific classes of objects are increasingly using 360° cameras, which can capture entire areas in a single picture. In this work, we present an interactive-based image processing, editing an...
Uložené v:
| Vydané v: | Graphical models Ročník 128; s. 101182 |
|---|---|
| Hlavní autori: | , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Elsevier Inc
01.07.2023
Elsevier |
| Predmet: | |
| ISSN: | 1524-0703 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Today’s Extended Reality (XR) applications that call for specific Diminished Reality (DR) strategies to hide specific classes of objects are increasingly using 360° cameras, which can capture entire areas in a single picture. In this work, we present an interactive-based image processing, editing and rendering system named SPIDER, that takes a spherical 360° indoor scene as input. The system is composed of a novel integrated deep learning architecture for extracting geometric and semantic information of full and empty rooms, based on gated and dilated convolutions, followed by a super-resolution module for improving the resolution of the color and depth signals. The obtained high resolution representations allow users to perform interactive exploration and basic editing operations on the reconstructed indoor scene, namely: (i) rendering of the scene in various modalities (point cloud, polygonal, wireframe) (ii) refurnishing (transferring portions of rooms) (iii) deferred shading through the usage of precomputed normal maps. These kinds of scene editing and manipulations can be used for assessing the inference from deep learning models and enable several Mixed Reality applications in areas such as furniture retails, interior designs, and real estates. Moreover, it can also be useful in data augmentation, arts, designs, and paintings. We report on the performance improvement of the various processing components on public domain spherical image indoor datasets.
[Display omitted] |
|---|---|
| AbstractList | Today’s Extended Reality (XR) applications that call for specific Diminished Reality (DR) strategies to hide specific classes of objects are increasingly using 360° cameras, which can capture entire areas in a single picture. In this work, we present an interactive-based image processing, editing and rendering system named SPIDER, that takes a spherical 360° indoor scene as input. The system is composed of a novel integrated deep learning architecture for extracting geometric and semantic information of full and empty rooms, based on gated and dilated convolutions, followed by a super-resolution module for improving the resolution of the color and depth signals. The obtained high resolution representations allow users to perform interactive exploration and basic editing operations on the reconstructed indoor scene, namely: (i) rendering of the scene in various modalities (point cloud, polygonal, wireframe) (ii) refurnishing (transferring portions of rooms) (iii) deferred shading through the usage of precomputed normal maps. These kinds of scene editing and manipulations can be used for assessing the inference from deep learning models and enable several Mixed Reality applications in areas such as furniture retails, interior designs, and real estates. Moreover, it can also be useful in data augmentation, arts, designs, and paintings. We report on the performance improvement of the various processing components on public domain spherical image indoor datasets.
[Display omitted] Today’s Extended Reality (XR) applications that call for specific Diminished Reality (DR) strategies to hide specific classes of objects are increasingly using 360° cameras, which can capture entire areas in a single picture. In this work, we present an interactive-based image processing, editing and rendering system named SPIDER, that takes a spherical 360° indoor scene as input. The system is composed of a novel integrated deep learning architecture for extracting geometric and semantic information of full and empty rooms, based on gated and dilated convolutions, followed by a super-resolution module for improving the resolution of the color and depth signals. The obtained high resolution representations allow users to perform interactive exploration and basic editing operations on the reconstructed indoor scene, namely: (i) rendering of the scene in various modalities (point cloud, polygonal, wireframe) (ii) refurnishing (transferring portions of rooms) (iii) deferred shading through the usage of precomputed normal maps. These kinds of scene editing and manipulations can be used for assessing the inference from deep learning models and enable several Mixed Reality applications in areas such as furniture retails, interior designs, and real estates. Moreover, it can also be useful in data augmentation, arts, designs, and paintings. We report on the performance improvement of the various processing components on public domain spherical image indoor datasets. |
| ArticleNumber | 101182 |
| Author | Pintore, G. Schneider, J. Tukur, M. Agus, M. Gobbetti, E. |
| Author_xml | – sequence: 1 givenname: M. orcidid: 0000-0003-1103-9659 surname: Tukur fullname: Tukur, M. organization: College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha, Qatar – sequence: 2 givenname: G. orcidid: 0000-0001-8944-1045 surname: Pintore fullname: Pintore, G. organization: Visual and Data-intensive Computing, CRS4, Italy – sequence: 3 givenname: E. orcidid: 0000-0003-0831-2458 surname: Gobbetti fullname: Gobbetti, E. organization: Visual and Data-intensive Computing, CRS4, Italy – sequence: 4 givenname: J. orcidid: 0000-0002-0546-2816 surname: Schneider fullname: Schneider, J. organization: College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha, Qatar – sequence: 5 givenname: M. orcidid: 0000-0003-2752-3525 surname: Agus fullname: Agus, M. email: magus@hbku.edu.qa organization: College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha, Qatar |
| BookMark | eNp9kctOxCAUhllo4vUFXPEAdqSUAjVujNdJTDRe1oTC6QxjCxOoGt9e6hgXLlwB_8n_JedjD2354AGho5LMSlLyk9VsMQQ7o4RWU1BKuoV2y5qygghS7aC9lFYk5zWju6h_ephfXj2e4nPcRT3AR4ivuAsRr2MwkJLzi2MM1o35grW3OYcE_vvphgFicu-Al26xLPIg9G-jCx6n9RKiM7rHztuQacmAh3SAtjvdJzj8OffRy_XV88VtcXd_M784vysMK8lYcAmMVpJw3XYNbyouS9E1hAEBAxUxRFbCcEFbmddojehs0wgtSdtSLVrQ1T6ab7g26JVaRzfo-KmCduo7CHGhdByd6UFZKpllUphWCtaQWteWMs41YbyWoqOZRTcsE0NKEbpfXknU5Fut1ORbTb7VxncuyT8l40Y9qRmjdv3_1bNNFbKgdwdRJePAm_wHEcyYN3D_1b8AAtif4g |
| CitedBy_id | crossref_primary_10_1016_j_jksuci_2024_101967 crossref_primary_10_3389_frvir_2025_1622605 crossref_primary_10_1016_j_cag_2024_103907 crossref_primary_10_1109_TVCG_2023_3320219 crossref_primary_10_3389_fdata_2023_1301812 crossref_primary_10_4018_IJCINI_378308 crossref_primary_10_1051_bioconf_20248601109 |
| Cites_doi | 10.1145/3478513.3480480 10.1145/3528223.3530074 10.1109/MCG.2021.3067999 10.1109/TPAMI.2020.3019967 10.1007/s41095-020-0162-z 10.1145/3072959.3073659 10.5244/C.34.120 10.1111/cgf.14021 10.1145/2980179.2980223 10.1109/TCE.2010.5505931 10.1109/TPAMI.2021.3076487 10.1145/3528223.3530173 10.1145/3528223.3530148 10.1145/3450626.3459812 10.1145/3329119 10.1109/LRA.2021.3058957 10.3390/ijgi9050330 |
| ContentType | Journal Article |
| Copyright | 2023 The Author(s) |
| Copyright_xml | – notice: 2023 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION DOA |
| DOI | 10.1016/j.gmod.2023.101182 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef DOAJ Open Access Full Text |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| ExternalDocumentID | oai_doaj_org_article_d284d487cb874905a5d2466a046587f2 10_1016_j_gmod_2023_101182 S1524070323000127 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 6I. 6TJ 7-5 71M 8P~ AACTN AAEDT AAEDW AAFTH AAFWJ AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AATTM AAXKI AAXUO AAYFN ABAOU ABBOA ABFNM ABJNI ABMAC ABWVN ABXDB ACDAQ ACGFS ACNNM ACRLP ACRPL ACZNC ADBBV ADEZE ADFGL ADGUI ADJOM ADMUD ADNMO ADVLN AEBSH AEIPS AEKER AENEX AEXQZ AFJKZ AFPKN AFTJW AGHFR AGUBO AGYEJ AHZHX AIALX AIEXJ AIGVJ AIKHN AITUG AKRWK ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU AOUOD ARUGR ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC BNPGV CAG COF CS3 DM4 DU5 EBS EFBJH EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ GROUPED_DOAJ HF~ HVGLF HZ~ IHE J1W KOM LG5 M41 MHUIS MO0 N9A O-L O9- OAUVE OZT P-8 P-9 PC. Q38 R2- RIG ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SSH SSV SSW SSZ T5K TN5 UHS XPP ZMT ZU3 ~02 ~G- 9DU AAYWO AAYXX ACLOT ACVFH ADCNI AEUPX AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP APXCP CITATION EFKBS EFLBG ~HD |
| ID | FETCH-LOGICAL-c410t-68e423806abf96936817f904e0ece30c0837c672b8001bc7fd997a80bb2a7bea3 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 7 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001041060600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1524-0703 |
| IngestDate | Fri Oct 03 12:31:36 EDT 2025 Tue Nov 18 22:08:52 EST 2025 Sat Nov 29 06:59:13 EST 2025 Sun Apr 06 06:53:30 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Reconstruction Spherical images Computing methodologies: Image-based rendering Indoor Environments Extended Reality Applied computing: Architecture (buildings) Deep Learning Super resolution |
| Language | English |
| License | This is an open access article under the CC BY-NC-ND license. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c410t-68e423806abf96936817f904e0ece30c0837c672b8001bc7fd997a80bb2a7bea3 |
| ORCID | 0000-0001-8944-1045 0000-0002-0546-2816 0000-0003-1103-9659 0000-0003-0831-2458 0000-0003-2752-3525 |
| OpenAccessLink | https://doaj.org/article/d284d487cb874905a5d2466a046587f2 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_d284d487cb874905a5d2466a046587f2 crossref_primary_10_1016_j_gmod_2023_101182 crossref_citationtrail_10_1016_j_gmod_2023_101182 elsevier_sciencedirect_doi_10_1016_j_gmod_2023_101182 |
| PublicationCentury | 2000 |
| PublicationDate | July 2023 2023-07-00 2023-07-01 |
| PublicationDateYYYYMMDD | 2023-07-01 |
| PublicationDate_xml | – month: 07 year: 2023 text: July 2023 |
| PublicationDecade | 2020 |
| PublicationTitle | Graphical models |
| PublicationYear | 2023 |
| Publisher | Elsevier Inc Elsevier |
| Publisher_xml | – name: Elsevier Inc – name: Elsevier |
| References | Stuccco (b49) 2022 Qi, Chen, Jia, Koltun (b40) 2018 Chen, Zhang, Finnie, Chalmers, Rhee (b10) 2022 Vázquez, Feixas, Sbert, Heidrich (b58) 2001 Fan, Lo, Pai, Hsu (b24) 2019; 52 Pirker, Dengel (b2) 2021; 41 Isola, Zhu, Zhou, Efros (b41) 2017 Wang, Lyu, Li, Zhang (b20) 2020; 6 Wei Zeng, Sezer Karaoglu, Theo Gevers, Pano2Scene: 3D Indoor Semantic Scene Reconstruction from a Single Panorama Image, in: Proc. BMVC, 2020. Pintore, Agus, Almansa, Gobbetti (b11) 2022 Tan, Chen, Chu, Chai, Liao, He, Yuan, Hua, Yu (b39) 2021 Armeni, Sax, Zamir, Savarese (b53) 2017 Ranftl, Lasinger, Hafner, Schindler, Koltun (b54) 2020; 44 Junayed, Sadeghzadeh, Islam, Wong, Aydın (b28) 2022 Guerrero, Hašan, Sunkavalli, Měch, Boubekeur, Mitra (b61) 2022; 41 Sun, Lu, Yu (b52) 2017; 24 Müller, Rousselle, Novák, Keller (b60) 2021; 40 Yu, Lin, Yang, Shen, Lu, Huang (b45) 2018 Jiang, Sheng, Zhu, Dong, Huang (b57) 2021; 6 Jang, Meuleman, Kang, Kim, Richardt, Kim (b1) 2022; 41 Zhi, Chen, Boyadzhiev, Kang, Hebert, Narasimhan (b9) 2022; 41 Wang, Xie, Dong, Shan (b19) 2021 Styldod (b48) 2022 Zhuang, Lu, Wang, Xiao, Wang (b29) 2022 Pintore, Agus, Almansa, Schneider, Gobbetti (b12) 2021 Chen, Wang, Zhou, Dong (b15) 2022 PadStyler (b50) 2022 Sun, Sun, Chen (b56) 2021 Deng, Wang, Xu, Guo, Song, Yang (b18) 2021 Yu, Koltun (b46) 2016 Kang, Yang, Yang, Cheng (b22) 2020; 9 Zhu, Abdal, Qin, Wonka (b16) 2020 da Silveira, Pinto, Murrugarra-Llerena, Jung (b23) 2022 Lee, Liu, Wu, Luo (b36) 2020 Eigen, Puhrsch, Fergus (b51) 2014; 27 Iizuka, Simo-Serra, Ishikawa (b44) 2017; 36 Chang, Dai, Funkhouser, Halber, Niessner, Savva, Song, Zeng, Zhang (b27) 2017 Pintore, Mura, Ganovelli, Fuentes-Perez, Pajarola, Gobbetti (b21) 2020; 39 Area, Yuan, Richardt (b34) 2022 Braud, Fernández, Hui (b5) 2022 Höllein, Johnson, Nießner (b59) 2022 Sun, Sun, Chen (b31) 2021 Peng, Zhang (b35) 2023 Pintore, Garro, Ganovelli, Gobbetti, Agus (b25) 2016 Gkitsas, Sterzentsenko, Zioulis, Albanis, Zarpalas (b43) 2021 Vazquez, Tan, Sadalgi (b7) 2021 Yang, Zhang, Reiß, Hu, Stiefelhagen (b3) 2021 Chen, Koltun (b38) 2017 Ranftl, Bochkovskiy, Koltun (b55) 2021 Xiong, Pulli (b4) 2010; 56 Yi, Tang, Azizi, Jang, Xu (b47) 2020 Pintore, Agus, Gobbetti (b26) 2020 Ma, Li, Zou, Liao, Tong, Zhang (b6) 2016; 35 Yun, Lee, Rhee (b30) 2022 Li, Wang, Huang, Pan, Yang (b33) 2022 Yu, Lin, Yang, Shen, Lu, Huang (b17) 2019 Park, Liu, Wang, Zhu (b37) 2019 Pintore, Almansa, Agus, Gobbetti (b13) 2021; 40 Sulaiman, Aziz, Bakar, Halili, Azuddin (b8) 2020 Zheng, Zhang, Li, Tang, Gao, Zhou (b14) 2020 Wang, Liu, Zhu, Tao, Kautz, Catanzaro (b42) 2018 Sun (10.1016/j.gmod.2023.101182_b52) 2017; 24 Vázquez (10.1016/j.gmod.2023.101182_b58) 2001 Zhu (10.1016/j.gmod.2023.101182_b16) 2020 Armeni (10.1016/j.gmod.2023.101182_b53) 2017 Kang (10.1016/j.gmod.2023.101182_b22) 2020; 9 Chen (10.1016/j.gmod.2023.101182_b38) 2017 Park (10.1016/j.gmod.2023.101182_b37) 2019 Yi (10.1016/j.gmod.2023.101182_b47) 2020 Pintore (10.1016/j.gmod.2023.101182_b26) 2020 Lee (10.1016/j.gmod.2023.101182_b36) 2020 Eigen (10.1016/j.gmod.2023.101182_b51) 2014; 27 Xiong (10.1016/j.gmod.2023.101182_b4) 2010; 56 Pirker (10.1016/j.gmod.2023.101182_b2) 2021; 41 Zhi (10.1016/j.gmod.2023.101182_b9) 2022; 41 Yu (10.1016/j.gmod.2023.101182_b46) 2016 10.1016/j.gmod.2023.101182_b32 Sun (10.1016/j.gmod.2023.101182_b56) 2021 Sun (10.1016/j.gmod.2023.101182_b31) 2021 Pintore (10.1016/j.gmod.2023.101182_b12) 2021 Höllein (10.1016/j.gmod.2023.101182_b59) 2022 Guerrero (10.1016/j.gmod.2023.101182_b61) 2022; 41 Müller (10.1016/j.gmod.2023.101182_b60) 2021; 40 Fan (10.1016/j.gmod.2023.101182_b24) 2019; 52 Ranftl (10.1016/j.gmod.2023.101182_b55) 2021 Chen (10.1016/j.gmod.2023.101182_b10) 2022 Area (10.1016/j.gmod.2023.101182_b34) 2022 Stuccco (10.1016/j.gmod.2023.101182_b49) 2022 Jiang (10.1016/j.gmod.2023.101182_b57) 2021; 6 Qi (10.1016/j.gmod.2023.101182_b40) 2018 Vazquez (10.1016/j.gmod.2023.101182_b7) 2021 Ranftl (10.1016/j.gmod.2023.101182_b54) 2020; 44 Zhuang (10.1016/j.gmod.2023.101182_b29) 2022 Deng (10.1016/j.gmod.2023.101182_b18) 2021 Pintore (10.1016/j.gmod.2023.101182_b13) 2021; 40 Ma (10.1016/j.gmod.2023.101182_b6) 2016; 35 Pintore (10.1016/j.gmod.2023.101182_b21) 2020; 39 Tan (10.1016/j.gmod.2023.101182_b39) 2021 Wang (10.1016/j.gmod.2023.101182_b42) 2018 Pintore (10.1016/j.gmod.2023.101182_b11) 2022 Styldod (10.1016/j.gmod.2023.101182_b48) 2022 Wang (10.1016/j.gmod.2023.101182_b20) 2020; 6 Li (10.1016/j.gmod.2023.101182_b33) 2022 Gkitsas (10.1016/j.gmod.2023.101182_b43) 2021 Chang (10.1016/j.gmod.2023.101182_b27) 2017 Sulaiman (10.1016/j.gmod.2023.101182_b8) 2020 Pintore (10.1016/j.gmod.2023.101182_b25) 2016 Yang (10.1016/j.gmod.2023.101182_b3) 2021 Wang (10.1016/j.gmod.2023.101182_b19) 2021 Peng (10.1016/j.gmod.2023.101182_b35) 2023 Junayed (10.1016/j.gmod.2023.101182_b28) 2022 PadStyler (10.1016/j.gmod.2023.101182_b50) 2022 Yu (10.1016/j.gmod.2023.101182_b17) 2019 Braud (10.1016/j.gmod.2023.101182_b5) 2022 Yu (10.1016/j.gmod.2023.101182_b45) 2018 Jang (10.1016/j.gmod.2023.101182_b1) 2022; 41 Zheng (10.1016/j.gmod.2023.101182_b14) 2020 Chen (10.1016/j.gmod.2023.101182_b15) 2022 Yun (10.1016/j.gmod.2023.101182_b30) 2022 Isola (10.1016/j.gmod.2023.101182_b41) 2017 da Silveira (10.1016/j.gmod.2023.101182_b23) 2022 Iizuka (10.1016/j.gmod.2023.101182_b44) 2017; 36 |
| References_xml | – start-page: 5212 year: 2022 end-page: 5221 ident: b28 article-title: HiMODE: A hybrid monocular omnidirectional depth estimation model publication-title: Proc. CVPR Workshops – volume: 40 start-page: 250:1 year: 2021 end-page: 250:12 ident: b13 article-title: Deep3DLayout: 3D reconstruction of an indoor layout from a spherical panoramic image publication-title: ACM Trans. Graph. – volume: 39 start-page: 667 year: 2020 end-page: 699 ident: b21 article-title: State-of-the-art in automatic 3D reconstruction of structured indoor environments publication-title: Comput. Graph. Forum – year: 2017 ident: b27 article-title: Matterport3D: Learning from RGB-D data in indoor environments publication-title: Proc. 3DV – volume: 6 start-page: 1519 year: 2021 end-page: 1526 ident: b57 article-title: Unifuse: Unidirectional fusion for 360 panorama depth estimation publication-title: IEEE Robot. Autom. Lett. – year: 2022 ident: b34 article-title: 360MonoDepth: High-resolution 360°monocular depth estimation publication-title: Conference on Computer Vision and Pattern Recognition – volume: 35 start-page: 1 year: 2016 end-page: 13 ident: b6 article-title: Action-driven 3D indoor scene evolution publication-title: ACM Trans. Graph. – start-page: 1 year: 2016 end-page: 9 ident: b25 article-title: Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps publication-title: Proc. WACV – start-page: 432 year: 2020 end-page: 448 ident: b26 article-title: AtlantaNet: Inferring the 3D indoor layout from a single 360 image beyond the Manhattan world assumption publication-title: Proc. ECCV – start-page: 12713 year: 2022 end-page: 12723 ident: b33 article-title: PhyIR: Physics-based inverse rendering for panoramic indoor images publication-title: Proc. CVPR – volume: 24 start-page: 1408 year: 2017 end-page: 1412 ident: b52 article-title: Weighted-to-spherically-uniform quality evaluation for omnidirectional video publication-title: IEEE Signal Process. Lett. – start-page: 7508 year: 2020 end-page: 7517 ident: b47 article-title: Contextual residual aggregation for ultra high-resolution image inpainting publication-title: Proc. CVPR – start-page: 1125 year: 2017 end-page: 1134 ident: b41 article-title: Image-to-image translation with conditional adversarial networks publication-title: Proc. CVPR – start-page: 12179 year: 2021 end-page: 12188 ident: b55 article-title: Vision transformers for dense prediction publication-title: Proceedings of the IEEE/CVF International Conference on Computer Vision – reference: Wei Zeng, Sezer Karaoglu, Theo Gevers, Pano2Scene: 3D Indoor Semantic Scene Reconstruction from a Single Panorama Image, in: Proc. BMVC, 2020. – start-page: 273 year: 2001 end-page: 280 ident: b58 article-title: Viewpoint selection using viewpoint entropy publication-title: VMV, Vol. 1 – start-page: 5549 year: 2020 end-page: 5558 ident: b36 article-title: Maskgan: Towards diverse and interactive facial image manipulation publication-title: Proc. CVPR – start-page: 1 year: 2019 ident: b37 article-title: GauGAN: semantic image synthesis with spatially adaptive normalization publication-title: ACM SIGGRAPH 2019 Real-Time Live! – year: 2021 ident: b39 article-title: Efficient semantic image synthesis via class-adaptive normalization publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – start-page: 2573 year: 2021 end-page: 2582 ident: b56 article-title: Hohonet: 360 indoor holistic understanding with latent horizontal features publication-title: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition – start-page: 3224 year: 2022 end-page: 3233 ident: b30 article-title: Improving 360 monocular depth estimation via non-local dense prediction transformer and joint supervised and self-supervised learning publication-title: Proc. AAAI Conference on Artificial Intelligence, Vol. 36, No. 3 – year: 2021 ident: b19 article-title: Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data publication-title: International Conference on Computer Vision Workshops – start-page: 5505 year: 2018 end-page: 5514 ident: b45 article-title: Generative image inpainting with contextual attention publication-title: Proc. CVPR – year: 2022 ident: b49 article-title: Virtual staging – year: 2022 ident: b48 article-title: Real estate virtual home staging – year: 2022 ident: b10 article-title: Casual 6-DoF: free-viewpoint panorama using a handheld 360 camera publication-title: IEEE Trans. Vis. Comput. Graphics – start-page: 3716 year: 2021 end-page: 3726 ident: b43 article-title: Panodr: Spherical panorama diminished reality for indoor scenes publication-title: Proc. CVPR Workshops – year: 2022 ident: b15 article-title: Activating more pixels in image super-resolution transformer – volume: 41 start-page: 1 year: 2022 end-page: 15 ident: b9 article-title: Semantically supervised appearance decomposition for virtual staging from a single panorama publication-title: ACM Trans. Graph. – volume: 41 start-page: 1 year: 2022 end-page: 12 ident: b61 article-title: MatFormer: a generative model for procedural materials publication-title: ACM Trans. Graph. – volume: 6 start-page: 3 year: 2020 end-page: 28 ident: b20 article-title: VR content creation and exploration with deep learning: A survey publication-title: Comput. Vis. Media – volume: 56 start-page: 298 year: 2010 end-page: 306 ident: b4 article-title: Fast panorama stitching for high-quality panoramic images on mobile phones publication-title: IEEE Trans. Consum. Electron. – start-page: 1 year: 2021 end-page: 5 ident: b7 article-title: Home studio: a mixed reality staging tool for interior design publication-title: Proc. CHI EA – volume: 36 year: 2017 ident: b44 article-title: Globally and locally consistent image completion publication-title: ACM Trans. Graph. – volume: 41 start-page: 1 year: 2022 end-page: 12 ident: b1 article-title: Egocentric scene reconstruction from an omnidirectional video publication-title: ACM Trans. Graph. – volume: 52 start-page: 1 year: 2019 end-page: 36 ident: b24 article-title: A survey on 360 video streaming: Acquisition, transmission, and display publication-title: ACM Comput. Surv. – start-page: 169 year: 2022 end-page: 175 ident: b5 article-title: Scaling-up ar: University campus as a physical-digital metaverse publication-title: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops – start-page: 1511 year: 2017 end-page: 1520 ident: b38 article-title: Photographic image synthesis with cascaded refinement networks publication-title: Proc. ICCV – start-page: 8798 year: 2018 end-page: 8807 ident: b42 article-title: High-resolution image synthesis and semantic manipulation with conditional gans publication-title: Proc. CVPR – start-page: 11536 year: 2021 end-page: 11545 ident: b12 article-title: SliceNet: deep dense depth estimation from a single indoor panorama using a slice-based representation publication-title: Proc. CVPR – start-page: 3115 year: 2023 end-page: 3124 ident: b35 article-title: High-resolution depth estimation for 360°panoramas through perspective and panoramic depth images registration publication-title: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision – start-page: 2573 year: 2021 end-page: 2582 ident: b31 article-title: HoHoNet: 360 indoor holistic understanding with latent horizontal features publication-title: Proc. CVPR – start-page: 8808 year: 2018 end-page: 8816 ident: b40 article-title: Semi-parametric image synthesis publication-title: Proc. CVPR – volume: 9 start-page: 330 year: 2020 ident: b22 article-title: A review of techniques for 3D reconstruction of indoor environments publication-title: ISPRS Int. J. Geo-Inf. – start-page: 1376 year: 2021 end-page: 1386 ident: b3 article-title: Capturing omni-range context for omnidirectional segmentation publication-title: Proc. CVPR – start-page: 6198 year: 2022 end-page: 6208 ident: b59 article-title: StyleMesh: Style transfer for indoor 3D scene reconstructions publication-title: Proc. CVPR – volume: 40 start-page: 1 year: 2021 end-page: 16 ident: b60 article-title: Real-time neural radiance caching for path tracing publication-title: ACM Trans. Graph. – volume: 41 start-page: 76 year: 2021 end-page: 89 ident: b2 article-title: The potential of 360 virtual reality videos and real VR for education—A literature review publication-title: IEEE Comput. Graph. Appl. – start-page: 221 year: 2020 end-page: 226 ident: b8 article-title: Matterport: virtual tour as a new marketing approach in real estate business during pandemic COVID-19 publication-title: Proc. IMDES – start-page: 519 year: 2020 end-page: 535 ident: b14 article-title: Structured3d: A large photo-realistic dataset for structured 3d modeling publication-title: Proc. ECCV – year: 2017 ident: b53 article-title: Joint 2d-3d-semantic data for indoor scene understanding – start-page: 5104 year: 2020 end-page: 5113 ident: b16 article-title: Sean: Image synthesis with semantic region-adaptive normalization publication-title: Proc. CVPR – start-page: 1 year: 2022 end-page: 11 ident: b11 article-title: Instant automatic emptying of panoramic indoor scenes publication-title: IEEE Trans. Vis. Comput. Graphics – year: 2022 ident: b50 article-title: 3D architectural rendering – start-page: 3653 year: 2022 end-page: 3661 ident: b29 article-title: ACDNet: Adaptively combined dilated convolution for monocular panorama depth estimation publication-title: Proc. AAAI Conference on Artificial Intelligence, Vol. 36, No. 3 – volume: 44 start-page: 1623 year: 2020 end-page: 1637 ident: b54 article-title: Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – year: 2016 ident: b46 article-title: Multi-scale context aggregation by dilated convolutions publication-title: Proc. ICLR – start-page: 4471 year: 2019 end-page: 4480 ident: b17 article-title: Free-form image inpainting with gated convolution publication-title: Proc. ICCV – volume: 27 year: 2014 ident: b51 article-title: Depth map prediction from a single image using a multi-scale deep network publication-title: Adv. Neural Inf. Process. Syst. – year: 2022 ident: b23 article-title: 3D scene geometry estimation from 360 imagery: A survey publication-title: ACM Comput. Surv. – start-page: 9189 year: 2021 end-page: 9198 ident: b18 article-title: LAU-net: Latitude adaptive upscaling network for omnidirectional image super-resolution publication-title: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition – start-page: 5212 year: 2022 ident: 10.1016/j.gmod.2023.101182_b28 article-title: HiMODE: A hybrid monocular omnidirectional depth estimation model – start-page: 3115 year: 2023 ident: 10.1016/j.gmod.2023.101182_b35 article-title: High-resolution depth estimation for 360°panoramas through perspective and panoramic depth images registration – start-page: 8808 year: 2018 ident: 10.1016/j.gmod.2023.101182_b40 article-title: Semi-parametric image synthesis – start-page: 273 year: 2001 ident: 10.1016/j.gmod.2023.101182_b58 article-title: Viewpoint selection using viewpoint entropy – volume: 40 start-page: 250:1 issue: 6 year: 2021 ident: 10.1016/j.gmod.2023.101182_b13 article-title: Deep3DLayout: 3D reconstruction of an indoor layout from a spherical panoramic image publication-title: ACM Trans. Graph. doi: 10.1145/3478513.3480480 – volume: 41 start-page: 1 issue: 4 year: 2022 ident: 10.1016/j.gmod.2023.101182_b1 article-title: Egocentric scene reconstruction from an omnidirectional video publication-title: ACM Trans. Graph. doi: 10.1145/3528223.3530074 – start-page: 12713 year: 2022 ident: 10.1016/j.gmod.2023.101182_b33 article-title: PhyIR: Physics-based inverse rendering for panoramic indoor images – volume: 41 start-page: 76 issue: 4 year: 2021 ident: 10.1016/j.gmod.2023.101182_b2 article-title: The potential of 360 virtual reality videos and real VR for education—A literature review publication-title: IEEE Comput. Graph. Appl. doi: 10.1109/MCG.2021.3067999 – start-page: 9189 year: 2021 ident: 10.1016/j.gmod.2023.101182_b18 article-title: LAU-net: Latitude adaptive upscaling network for omnidirectional image super-resolution – start-page: 1 year: 2022 ident: 10.1016/j.gmod.2023.101182_b11 article-title: Instant automatic emptying of panoramic indoor scenes publication-title: IEEE Trans. Vis. Comput. Graphics – volume: 44 start-page: 1623 issue: 3 year: 2020 ident: 10.1016/j.gmod.2023.101182_b54 article-title: Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer publication-title: IEEE Trans. Pattern Anal. Mach. Intell. doi: 10.1109/TPAMI.2020.3019967 – volume: 6 start-page: 3 issue: 1 year: 2020 ident: 10.1016/j.gmod.2023.101182_b20 article-title: VR content creation and exploration with deep learning: A survey publication-title: Comput. Vis. Media doi: 10.1007/s41095-020-0162-z – start-page: 4471 year: 2019 ident: 10.1016/j.gmod.2023.101182_b17 article-title: Free-form image inpainting with gated convolution – start-page: 5549 year: 2020 ident: 10.1016/j.gmod.2023.101182_b36 article-title: Maskgan: Towards diverse and interactive facial image manipulation – year: 2022 ident: 10.1016/j.gmod.2023.101182_b15 – start-page: 5104 year: 2020 ident: 10.1016/j.gmod.2023.101182_b16 article-title: Sean: Image synthesis with semantic region-adaptive normalization – year: 2022 ident: 10.1016/j.gmod.2023.101182_b50 – year: 2017 ident: 10.1016/j.gmod.2023.101182_b27 article-title: Matterport3D: Learning from RGB-D data in indoor environments – volume: 36 issue: 4 year: 2017 ident: 10.1016/j.gmod.2023.101182_b44 article-title: Globally and locally consistent image completion publication-title: ACM Trans. Graph. doi: 10.1145/3072959.3073659 – year: 2022 ident: 10.1016/j.gmod.2023.101182_b48 – ident: 10.1016/j.gmod.2023.101182_b32 doi: 10.5244/C.34.120 – start-page: 432 year: 2020 ident: 10.1016/j.gmod.2023.101182_b26 article-title: AtlantaNet: Inferring the 3D indoor layout from a single 360 image beyond the Manhattan world assumption – start-page: 1 year: 2016 ident: 10.1016/j.gmod.2023.101182_b25 article-title: Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps – start-page: 169 year: 2022 ident: 10.1016/j.gmod.2023.101182_b5 article-title: Scaling-up ar: University campus as a physical-digital metaverse – start-page: 2573 year: 2021 ident: 10.1016/j.gmod.2023.101182_b56 article-title: Hohonet: 360 indoor holistic understanding with latent horizontal features – year: 2022 ident: 10.1016/j.gmod.2023.101182_b10 article-title: Casual 6-DoF: free-viewpoint panorama using a handheld 360 camera publication-title: IEEE Trans. Vis. Comput. Graphics – year: 2017 ident: 10.1016/j.gmod.2023.101182_b53 – volume: 39 start-page: 667 issue: 2 year: 2020 ident: 10.1016/j.gmod.2023.101182_b21 article-title: State-of-the-art in automatic 3D reconstruction of structured indoor environments publication-title: Comput. Graph. Forum doi: 10.1111/cgf.14021 – start-page: 1376 year: 2021 ident: 10.1016/j.gmod.2023.101182_b3 article-title: Capturing omni-range context for omnidirectional segmentation – year: 2022 ident: 10.1016/j.gmod.2023.101182_b23 article-title: 3D scene geometry estimation from 360 imagery: A survey publication-title: ACM Comput. Surv. – year: 2022 ident: 10.1016/j.gmod.2023.101182_b49 – volume: 35 start-page: 1 issue: 6 year: 2016 ident: 10.1016/j.gmod.2023.101182_b6 article-title: Action-driven 3D indoor scene evolution publication-title: ACM Trans. Graph. doi: 10.1145/2980179.2980223 – start-page: 1125 year: 2017 ident: 10.1016/j.gmod.2023.101182_b41 article-title: Image-to-image translation with conditional adversarial networks – volume: 56 start-page: 298 issue: 2 year: 2010 ident: 10.1016/j.gmod.2023.101182_b4 article-title: Fast panorama stitching for high-quality panoramic images on mobile phones publication-title: IEEE Trans. Consum. Electron. doi: 10.1109/TCE.2010.5505931 – year: 2021 ident: 10.1016/j.gmod.2023.101182_b39 article-title: Efficient semantic image synthesis via class-adaptive normalization publication-title: IEEE Trans. Pattern Anal. Mach. Intell. doi: 10.1109/TPAMI.2021.3076487 – volume: 41 start-page: 1 issue: 4 year: 2022 ident: 10.1016/j.gmod.2023.101182_b61 article-title: MatFormer: a generative model for procedural materials publication-title: ACM Trans. Graph. doi: 10.1145/3528223.3530173 – start-page: 519 year: 2020 ident: 10.1016/j.gmod.2023.101182_b14 article-title: Structured3d: A large photo-realistic dataset for structured 3d modeling – start-page: 3716 year: 2021 ident: 10.1016/j.gmod.2023.101182_b43 article-title: Panodr: Spherical panorama diminished reality for indoor scenes – volume: 41 start-page: 1 issue: 4 year: 2022 ident: 10.1016/j.gmod.2023.101182_b9 article-title: Semantically supervised appearance decomposition for virtual staging from a single panorama publication-title: ACM Trans. Graph. doi: 10.1145/3528223.3530148 – year: 2021 ident: 10.1016/j.gmod.2023.101182_b19 article-title: Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data – volume: 27 year: 2014 ident: 10.1016/j.gmod.2023.101182_b51 article-title: Depth map prediction from a single image using a multi-scale deep network publication-title: Adv. Neural Inf. Process. Syst. – start-page: 3653 year: 2022 ident: 10.1016/j.gmod.2023.101182_b29 article-title: ACDNet: Adaptively combined dilated convolution for monocular panorama depth estimation – volume: 40 start-page: 1 issue: 4 year: 2021 ident: 10.1016/j.gmod.2023.101182_b60 article-title: Real-time neural radiance caching for path tracing publication-title: ACM Trans. Graph. doi: 10.1145/3450626.3459812 – volume: 52 start-page: 1 issue: 4 year: 2019 ident: 10.1016/j.gmod.2023.101182_b24 article-title: A survey on 360 video streaming: Acquisition, transmission, and display publication-title: ACM Comput. Surv. doi: 10.1145/3329119 – year: 2016 ident: 10.1016/j.gmod.2023.101182_b46 article-title: Multi-scale context aggregation by dilated convolutions – start-page: 221 year: 2020 ident: 10.1016/j.gmod.2023.101182_b8 article-title: Matterport: virtual tour as a new marketing approach in real estate business during pandemic COVID-19 – start-page: 7508 year: 2020 ident: 10.1016/j.gmod.2023.101182_b47 article-title: Contextual residual aggregation for ultra high-resolution image inpainting – start-page: 12179 year: 2021 ident: 10.1016/j.gmod.2023.101182_b55 article-title: Vision transformers for dense prediction – start-page: 3224 year: 2022 ident: 10.1016/j.gmod.2023.101182_b30 article-title: Improving 360 monocular depth estimation via non-local dense prediction transformer and joint supervised and self-supervised learning – start-page: 1511 year: 2017 ident: 10.1016/j.gmod.2023.101182_b38 article-title: Photographic image synthesis with cascaded refinement networks – volume: 6 start-page: 1519 issue: 2 year: 2021 ident: 10.1016/j.gmod.2023.101182_b57 article-title: Unifuse: Unidirectional fusion for 360 panorama depth estimation publication-title: IEEE Robot. Autom. Lett. doi: 10.1109/LRA.2021.3058957 – start-page: 2573 year: 2021 ident: 10.1016/j.gmod.2023.101182_b31 article-title: HoHoNet: 360 indoor holistic understanding with latent horizontal features – start-page: 1 year: 2021 ident: 10.1016/j.gmod.2023.101182_b7 article-title: Home studio: a mixed reality staging tool for interior design – start-page: 11536 year: 2021 ident: 10.1016/j.gmod.2023.101182_b12 article-title: SliceNet: deep dense depth estimation from a single indoor panorama using a slice-based representation – start-page: 1 year: 2019 ident: 10.1016/j.gmod.2023.101182_b37 article-title: GauGAN: semantic image synthesis with spatially adaptive normalization – year: 2022 ident: 10.1016/j.gmod.2023.101182_b34 article-title: 360MonoDepth: High-resolution 360°monocular depth estimation – start-page: 6198 year: 2022 ident: 10.1016/j.gmod.2023.101182_b59 article-title: StyleMesh: Style transfer for indoor 3D scene reconstructions – start-page: 5505 year: 2018 ident: 10.1016/j.gmod.2023.101182_b45 article-title: Generative image inpainting with contextual attention – volume: 24 start-page: 1408 issue: 9 year: 2017 ident: 10.1016/j.gmod.2023.101182_b52 article-title: Weighted-to-spherically-uniform quality evaluation for omnidirectional video publication-title: IEEE Signal Process. Lett. – start-page: 8798 year: 2018 ident: 10.1016/j.gmod.2023.101182_b42 article-title: High-resolution image synthesis and semantic manipulation with conditional gans – volume: 9 start-page: 330 issue: 5 year: 2020 ident: 10.1016/j.gmod.2023.101182_b22 article-title: A review of techniques for 3D reconstruction of indoor environments publication-title: ISPRS Int. J. Geo-Inf. doi: 10.3390/ijgi9050330 |
| SSID | ssj0011542 |
| Score | 2.395358 |
| Snippet | Today’s Extended Reality (XR) applications that call for specific Diminished Reality (DR) strategies to hide specific classes of objects are increasingly using... |
| SourceID | doaj crossref elsevier |
| SourceType | Open Website Enrichment Source Index Database Publisher |
| StartPage | 101182 |
| SubjectTerms | Deep Learning Extended Reality Indoor Environments Spherical images Super resolution |
| Title | SPIDER: A framework for processing, editing and presenting immersive high-resolution spherical indoor scenes |
| URI | https://dx.doi.org/10.1016/j.gmod.2023.101182 https://doaj.org/article/d284d487cb874905a5d2466a046587f2 |
| Volume | 128 |
| WOSCitedRecordID | wos001041060600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals issn: 1524-0703 databaseCode: DOA dateStart: 20230101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.doaj.org/ omitProxy: false ssIdentifier: ssj0011542 providerName: Directory of Open Access Journals – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 1524-0703 databaseCode: AIEXJ dateStart: 20000101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0011542 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELZQYYAB8RTlUXlggwjnaZutQCvKUFU8pG6Rn6iopFVb-P2c46QqS1kY41wu0fnk-y4-f4fQZWzBU5SlQcilDRJOeCCMgGQl1lqLyKhE-2YTtN9nwyEfrLT6cjVhnh7YG-5Gw_qpAVUrySioSkWqoyTLBOR1KaO2XH0J5XUyVe0fADAo9znTKAmcU1fHZXxl1_vnxHGERrEbCFn0KySVzP0rkWkl2nT30G4FE3Hbf94-2jDFAdpZIQ88ROOXQe-h83yL29jWFVYYICie-tp_ELrGEJpcXTMWhcZTf9LIXY7K39Ww0GFHVxzAjcoD8dzRDLiJw5CsT0CbY3sy8yP01u283j8GVe-EQCUhWQQZMwCUGMmEtDzjccZCajlJDDHKxEQB8qIqo5EEwBhKRa3mnApGpIwElUbEx6hRTApzgrBRKVMUZK0KE0GJMDwSqWRal03nVROFtflyVRGLu_4W47yuIPvInclzZ_Lcm7yJrpbPTD2txlrpOzcrS0lHiV0OgKPklaPkfzlKE6X1nOYVuvCoAVSN1rz89D9efoa2nUpf53uOGovZl7lAW-p7MZrPWmiz3esMn1qlD_8AB27yZA |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SPIDER%3A+A+framework+for+processing%2C+editing+and+presenting+immersive+high-resolution+spherical+indoor+scenes&rft.jtitle=Graphical+models&rft.au=M.+Tukur&rft.au=G.+Pintore&rft.au=E.+Gobbetti&rft.au=J.+Schneider&rft.date=2023-07-01&rft.pub=Elsevier&rft.issn=1524-0703&rft.volume=128&rft.spage=101182&rft_id=info:doi/10.1016%2Fj.gmod.2023.101182&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_d284d487cb874905a5d2466a046587f2 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1524-0703&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1524-0703&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1524-0703&client=summon |