DualGNN: Dual Graph Neural Network for Multimedia Recommendation
One of the important factors affecting micro-video recommender systems is to model the multi-modal user preference on the micro-video. Despite the remarkable performance of prior arts, they are still limited by fusing the user preference derived from different modalities in a unified manner, ignorin...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on multimedia Jg. 25; S. 1074 - 1084 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Piscataway
IEEE
2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 1520-9210, 1941-0077 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | One of the important factors affecting micro-video recommender systems is to model the multi-modal user preference on the micro-video. Despite the remarkable performance of prior arts, they are still limited by fusing the user preference derived from different modalities in a unified manner, ignoring the users tend to place different emphasis on different modalities. Furthermore, modality-missing is ubiquity and unavoidable in the micro-video recommendation, some modalities information of micro-videos are lacked in many cases, which negatively affects the multi-modal fusion operations. To overcome these disadvantages, we propose a novel framework for the micro-video recommendation, dubbed Dual Graph Neural Network (DualGNN), upon the user-microvideo bipartite and user co-occurrence graphs, which leverages the correlation between users to collaboratively mine the particular fusion pattern for each user. Specifically, we first introduce a single-modal representation learning module, which performs graph operations on the user-microvideo graph in each modality to capture single-modal user preferences on different modalities. And then, we devise a multi-modal representation learning module to explicitly model the user's attentions over different modalities and inductively learn the multi-modal user preference. Finally, we propose a prediction module to rank the potential micro-videos for users. Extensive experiments on two public datasets demonstrate the significant superiority of our DualGNN over state-of-the-arts methods. |
|---|---|
| AbstractList | One of the important factors affecting micro-video recommender systems is to model the multi-modal user preference on the micro-video. Despite the remarkable performance of prior arts, they are still limited by fusing the user preference derived from different modalities in a unified manner, ignoring the users tend to place different emphasis on different modalities. Furthermore, modality-missing is ubiquity and unavoidable in the micro-video recommendation, some modalities information of micro-videos are lacked in many cases, which negatively affects the multi-modal fusion operations. To overcome these disadvantages, we propose a novel framework for the micro-video recommendation, dubbed Dual Graph Neural Network (DualGNN), upon the user-microvideo bipartite and user co-occurrence graphs, which leverages the correlation between users to collaboratively mine the particular fusion pattern for each user. Specifically, we first introduce a single-modal representation learning module, which performs graph operations on the user-microvideo graph in each modality to capture single-modal user preferences on different modalities. And then, we devise a multi-modal representation learning module to explicitly model the user’s attentions over different modalities and inductively learn the multi-modal user preference. Finally, we propose a prediction module to rank the potential micro-videos for users. Extensive experiments on two public datasets demonstrate the significant superiority of our DualGNN over state-of-the-arts methods. |
| Author | Song, Xuemeng Wu, Jianlong Wang, Qifan Wei, Yinwei Yin, Jianhua Nie, Liqiang |
| Author_xml | – sequence: 1 givenname: Qifan orcidid: 0000-0002-1237-366X surname: Wang fullname: Wang, Qifan email: wqf@mail.sdu.edu.cn organization: College of Computer Science and Technology, Shandong University, Qingdao, China – sequence: 2 givenname: Yinwei orcidid: 0000-0003-1791-3159 surname: Wei fullname: Wei, Yinwei email: weiyinwei@hotmail.com organization: School of Computing, National University of Singapore, Singapore – sequence: 3 givenname: Jianhua orcidid: 0000-0002-4611-2986 surname: Yin fullname: Yin, Jianhua email: jhyin@sdu.edu.cn organization: College of Computer Science and Technology, Shandong University, Qingdao, China – sequence: 4 givenname: Jianlong orcidid: 0000-0003-0247-5221 surname: Wu fullname: Wu, Jianlong email: jlwu1992@sdu.edu.cn organization: College of Computer Science and Technology, Shandong University, Qingdao, China – sequence: 5 givenname: Xuemeng orcidid: 0000-0002-5274-4197 surname: Song fullname: Song, Xuemeng email: sxmustc@gmail.com organization: College of Computer Science and Technology, Shandong University, Qingdao, China – sequence: 6 givenname: Liqiang orcidid: 0000-0003-1476-0273 surname: Nie fullname: Nie, Liqiang email: nieliqiang@gmail.com organization: College of Computer Science and Technology, Shandong University, Qingdao, China |
| BookMark | eNp9kEFLw0AQhRepYFu9C14CnlNnNtlk15NStQptBKnnZbPZYGqSrZsE8d-7tcWDB2HgvYH5ZoY3IaPWtoaQc4QZIoir9Wo1o0BxFmHEqeBHZIwixhAgTUfeMwqhoAgnZNJ1GwCMGaRjcnM3qHqRZdfBzgQLp7ZvQWYG55vM9J_WvQeldcFqqPuqMUWlghejbdOYtlB9ZdtTclyqujNnB52S14f79fwxXD4vnua3y1BTgX2oY8ZTpaFUigkm8jwxORpRaM2x5AhcK2C8jJCXsTEmL1LFdKFVRD2kvE7J5X7v1tmPwXS93NjBtf6kpKkQnEWRrylJ9lPa2a5zppS66n_-7J2qaokgd2lJn5bcpSUPaXkQ_oBbVzXKff2HXOyRyj_8Oy6ShCaMRd9WeHdQ |
| CODEN | ITMUF8 |
| CitedBy_id | crossref_primary_10_1016_j_inffus_2025_103168 crossref_primary_10_1109_TKDE_2024_3424268 crossref_primary_10_1109_TNSE_2024_3383440 crossref_primary_10_3390_systems13010057 crossref_primary_10_1007_s00521_025_11372_6 crossref_primary_10_1007_s10844_024_00848_x crossref_primary_10_3390_app142210187 crossref_primary_10_1007_s11263_024_02128_1 crossref_primary_10_1109_TCSS_2024_3490801 crossref_primary_10_1109_ACCESS_2025_3573204 crossref_primary_10_1007_s10489_024_06038_0 crossref_primary_10_1016_j_knosys_2025_114020 crossref_primary_10_1016_j_knosys_2025_113496 crossref_primary_10_1371_journal_pone_0287927 crossref_primary_10_1109_TKDE_2024_3493374 crossref_primary_10_1038_s41598_025_96462_0 crossref_primary_10_1145_3711855 crossref_primary_10_1109_TKDE_2023_3309995 crossref_primary_10_1016_j_knosys_2025_113766 crossref_primary_10_1109_TMM_2024_3361729 crossref_primary_10_1016_j_knosys_2023_111340 crossref_primary_10_1109_TSC_2025_3556640 crossref_primary_10_1109_TMM_2024_3382889 crossref_primary_10_4018_JOEUC_368009 crossref_primary_10_1007_s10115_025_02536_w crossref_primary_10_1007_s00530_024_01548_w crossref_primary_10_1038_s41598_025_96458_w crossref_primary_10_1109_TETCI_2024_3358190 crossref_primary_10_1145_3762665 crossref_primary_10_1007_s10115_024_02309_x crossref_primary_10_1109_TBDATA_2024_3426355 crossref_primary_10_1016_j_eswa_2025_128929 crossref_primary_10_1016_j_knosys_2025_113035 crossref_primary_10_1109_TETCI_2024_3386774 crossref_primary_10_1109_TMM_2024_3384058 crossref_primary_10_1016_j_compeleceng_2024_109213 crossref_primary_10_1145_3662738 crossref_primary_10_1016_j_iot_2025_101769 crossref_primary_10_1109_TMM_2024_3369875 crossref_primary_10_1007_s11704_024_3939_x crossref_primary_10_1145_3682075 crossref_primary_10_3390_e27010056 crossref_primary_10_1007_s10844_025_00946_4 crossref_primary_10_1088_2632_2153_ad66af crossref_primary_10_1016_j_inffus_2023_101989 crossref_primary_10_1109_ACCESS_2022_3174212 crossref_primary_10_1016_j_ipm_2025_104075 crossref_primary_10_1145_3715876 crossref_primary_10_1007_s10115_025_02456_9 crossref_primary_10_1007_s44443_025_00026_1 crossref_primary_10_1109_TCSVT_2024_3416294 |
| Cites_doi | 10.1109/CVPR.2019.01028 10.1109/TCSVT.2019.2962216 10.1145/3442381.3449986 10.1145/3394171.3413556 10.3390/rs10010052 10.24963/ijcai.2019/592 10.1145/3394171.3414012 10.1145/3308558.3313513 10.1109/ICASSP.2017.7952132 10.1609/aaai.v34i01.5330 10.1145/2964284.2964307 10.1145/3343031.3350950 10.1109/TIP.2019.2923608 10.1609/aaai.v30i1.9973 10.1145/3331184.3331267 10.1109/ICME51207.2021.9428201 10.1016/j.ipm.2020.102277 10.1109/TIP.2020.3002083 10.1145/3240508.3240546 10.1109/TMM.2021.3088307 10.1109/ICIP.2019.8803025 10.1145/3394171.3413653 10.1145/3343031.3351034 10.1145/3447239 10.1109/TNNLS.2022.3163789 10.1109/CVPR.2016.90 10.1145/3397271.3401063 10.1109/ICME51207.2021.9428120 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TMM.2021.3138298 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1941-0077 |
| EndPage | 1084 |
| ExternalDocumentID | 10_1109_TMM_2021_3138298 9662655 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Natural Science Foundation of Shandong Province; Shandong Provincial Natural Science Foundation grantid: ZR2019QF001 funderid: 10.13039/501100007129 – fundername: National Natural Science Foundation of China grantid: 62172261; 61802231 funderid: 10.13039/501100001809 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS TN5 VH1 ZY4 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c291t-c4587ac0faa5959bb6eb1e9dcc81f8108ca058f318f4eeebd7a5cdca32ac0aca3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 100 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000970791100004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1520-9210 |
| IngestDate | Sun Jun 29 15:39:34 EDT 2025 Tue Nov 18 22:43:53 EST 2025 Sat Nov 29 03:10:08 EST 2025 Wed Aug 27 02:21:33 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-c4587ac0faa5959bb6eb1e9dcc81f8108ca058f318f4eeebd7a5cdca32ac0aca3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-1791-3159 0000-0003-0247-5221 0000-0003-1476-0273 0000-0002-1237-366X 0000-0002-5274-4197 0000-0002-4611-2986 |
| PQID | 2799853353 |
| PQPubID | 75737 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_9662655 crossref_citationtrail_10_1109_TMM_2021_3138298 proquest_journals_2799853353 crossref_primary_10_1109_TMM_2021_3138298 |
| PublicationCentury | 2000 |
| PublicationDate | 20230000 2023-00-00 20230101 |
| PublicationDateYYYYMMDD | 2023-01-01 |
| PublicationDate_xml | – year: 2023 text: 20230000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on multimedia |
| PublicationTitleAbbrev | TMM |
| PublicationYear | 2023 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 ref14 ref31 ref30 ref11 ref33 ref10 Glorot (ref34) 2010 Kingma (ref35) 2015 ref2 ref1 ref17 Couprie (ref24) ref19 ref18 Rendle (ref29) 2009 Arora (ref32) 2016 Kipf (ref16) 2017 ref23 ref26 ref25 ref20 ref22 ref28 ref27 ref8 ref7 ref9 Berg (ref21) 2017 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref14 doi: 10.1109/CVPR.2019.01028 – ident: ref15 doi: 10.1109/TCSVT.2019.2962216 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2016 ident: ref32 article-title: A simple but tough-to-beat baseline for sentence embeddings – ident: ref23 doi: 10.1145/3442381.3449986 – ident: ref2 doi: 10.1145/3394171.3413556 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2015 ident: ref35 article-title: Adam: A method for stochastic optimization – ident: ref28 doi: 10.3390/rs10010052 – ident: ref33 doi: 10.24963/ijcai.2019/592 – ident: ref18 doi: 10.1145/3394171.3414012 – ident: ref5 doi: 10.1145/3308558.3313513 – ident: ref31 doi: 10.1109/ICASSP.2017.7952132 – ident: ref8 doi: 10.1609/aaai.v34i01.5330 – ident: ref7 doi: 10.1145/2964284.2964307 – ident: ref10 doi: 10.1145/3343031.3350950 – start-page: 452 volume-title: Proc. Conf. Uncertainty Artif. Intell. year: 2009 ident: ref29 article-title: BPR: Bayesian personalized ranking from implicit feedback – ident: ref26 doi: 10.1109/TIP.2019.2923608 – ident: ref12 doi: 10.1609/aaai.v30i1.9973 – ident: ref22 doi: 10.1145/3331184.3331267 – ident: ref27 doi: 10.1109/ICME51207.2021.9428201 – ident: ref11 doi: 10.1016/j.ipm.2020.102277 – ident: ref20 doi: 10.1109/TIP.2020.3002083 – ident: ref3 doi: 10.1145/3240508.3240546 – ident: ref19 doi: 10.1109/TMM.2021.3088307 – start-page: 249 volume-title: Proc. Int. Conf. Artif. Intell. Statist. year: 2010 ident: ref34 article-title: Understanding the difficulty of training deep feedforward neural networks – ident: ref24 article-title: Indoor semantic segmentation using depth information – ident: ref25 doi: 10.1109/ICIP.2019.8803025 – start-page: 1 volume-title: Proc. Int. Conf. Learn. Representations year: 2017 ident: ref16 article-title: Semi-supervised classification with graph convolutional networks – ident: ref1 doi: 10.1145/3394171.3413653 – ident: ref6 doi: 10.1145/3343031.3351034 – ident: ref17 doi: 10.1145/3447239 – start-page: 1 volume-title: Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining year: 2017 ident: ref21 article-title: Graph convolutional matrix completion – ident: ref4 doi: 10.1109/TNNLS.2022.3163789 – ident: ref30 doi: 10.1109/CVPR.2016.90 – ident: ref9 doi: 10.1145/3397271.3401063 – ident: ref13 doi: 10.1109/ICME51207.2021.9428120 |
| SSID | ssj0014507 |
| Score | 2.6710408 |
| Snippet | One of the important factors affecting micro-video recommender systems is to model the multi-modal user preference on the micro-video. Despite the remarkable... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1074 |
| SubjectTerms | Acoustics graph neural network Graph neural networks Learning Micro-video recommender systems Modules multi-modal fusion Multimedia Neural networks Preferences Recommender systems Representation learning Representations Task analysis Video Videos Visualization |
| Title | DualGNN: Dual Graph Neural Network for Multimedia Recommendation |
| URI | https://ieeexplore.ieee.org/document/9662655 https://www.proquest.com/docview/2799853353 |
| Volume | 25 |
| WOSCitedRecordID | wos000970791100004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Xplore customDbUrl: eissn: 1941-0077 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014507 issn: 1520-9210 databaseCode: RIE dateStart: 19990101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDLbGxAEODDYQg4Fy4IJEWfrIknACARuXVRyGtFuVpqmENDa0B78fJ30IBELi1FSK1dauYzuO_QFcoFkREdPcS3OqvMjXqaf8wHgs49pXHJ18Qx3YBI9jMZ3K5wZc1bUwxhh3-Mxc26HL5WcLvbFbZX10zYMBY1uwxfmgqNWqMwYRc6XRaI6oJzGOqVKSVPYn4zEGgoGP8WkoAim-mSCHqfJjIXbWZdj633vtw17pRZK7QuwH0DDzNrQqhAZSKmwbdr-0G-zA7cNGzUZxfEPsgIxsr2piu3PgTVwcByfowxJXlOtKSoiNTt_w6QXy0iG8DB8n909eiaDg6UD6a09HTHClaa4Uk0ym6QCXZiMzrYWfC58KrSgTOep1HuFXpRlXTGdahQESKbweQXO-mJtjINT4JgjzjBmFBk3ntutM6OqwqUIdT7vQr5ia6LK9uEW5mCUuzKAyQTEkVgxJKYYuXNYU70VrjT_mdizb63klx7vQq-SWlLq3SgKOISR6sSw8-Z3qFHYsaHyxkdKD5nq5MWewrT_Wr6vlufutPgHxtMn- |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS8MwED_mFNQHp5vi_MyDL4J1adqsiU-Kuk10xYcJeytpmoKgU_bh3-8l7YaiCD41hRxt73q5u1zufgAnaFZEyHXkpTlVXujr1FM-Mx7PIu2rCJ18Qx3YRBTHYjiUjxU4W9TCGGPc4TNzbocul5-96ZndKmuha87anC_BMg9DRotqrUXOIOSuOBoNEvUkRjLzpCSVrUG_j6Eg8zFCDQST4psRcqgqP5ZiZ186tf-92SZslH4kuSoEvwUVM6pDbY7RQEqVrcP6l4aDDbi8mamXbhxfEDsgXdutmtj-HHgTFwfCCXqxxJXluqISYuPTV3x6gb20DU-d28F1zysxFDzNpD_1dMhFpDTNleKSyzRt4-JsZKa18HPhU6EV5SJHzc5D_Ko0ixTXmVYBQyKF1x2ojt5GZhcINb5hQZ5xo9Ck6dz2nQlcJTZVqOVpE1pzpia6bDBucS5eEhdoUJmgGBIrhqQUQxNOFxTvRXONP-Y2LNsX80qON-FgLrek1L5JwiIMItGP5cHe71THsNob9B-Sh7v4fh_WLIR8sa1yANXpeGYOYUV_TJ8n4yP3i30CnjDNRQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DualGNN%3A+Dual+Graph+Neural+Network+for+Multimedia+Recommendation&rft.jtitle=IEEE+transactions+on+multimedia&rft.au=Wang%2C+Qifan&rft.au=Wei%2C+Yinwei&rft.au=Yin%2C+Jianhua&rft.au=Wu%2C+Jianlong&rft.date=2023&rft.issn=1520-9210&rft.eissn=1941-0077&rft.volume=25&rft.spage=1074&rft.epage=1084&rft_id=info:doi/10.1109%2FTMM.2021.3138298&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TMM_2021_3138298 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1520-9210&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1520-9210&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1520-9210&client=summon |