Multi-Modal Self-Supervised Learning Algorithm-Based Product Recommendation
With the rise of e-commerce, personalized recommendation algorithms have received much attention in recent years. Meanwhile, multimodal recommendation algorithms have become the next competitive track of personalized recommendation. The current mainstream recommendation algorithms (e.g., NeuMF, NGCF...
Saved in:
| Published in: | IEEE transactions on automation science and engineering Vol. 22; pp. 15371 - 15380 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
IEEE
2025
|
| Subjects: | |
| ISSN: | 1545-5955, 1558-3783 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | With the rise of e-commerce, personalized recommendation algorithms have received much attention in recent years. Meanwhile, multimodal recommendation algorithms have become the next competitive track of personalized recommendation. The current mainstream recommendation algorithms (e.g., NeuMF, NGCF) rely too much on empirical data and do not effectively utilize multimodal information to model user preferences, while there is the problem of sparsity of user preference data. Secondly, they are still limited to using the same weights to fuse the item features of different modalities, and have the problem of inaccurate recommendation. For this reason, this paper proposes Multi-Modal Self-Supervised Learning Algorithm Based Product Recommendation (MMSLRec). 1) Data augmentation by self-supervised adversarial perturbation generation learning is performed to supplement the missing interaction information of tailed products to solve the common data sparsity problem in product recommendation. 2) A multimodal representation learning module is introduced to capture deep features in image and text modalities using AlexNet and Bert, respectively, in order to avoid the recommendation accuracy degradation problem caused by insufficient data feature learning. 3) Learning the user's perceived strength in different modalities, using the user co-occurrence matrix to fuse the features and make recommendations. Experiments on Amazon-Electronics, Amazon-Clothing, and Amazon-Baby, with different degrees of data sparsity verify the effectiveness of our MMSLRec in solving the data sparsity problem and the multimodal user preference modeling problem, and compared to the state-of-the-art baseline recommendation model on the three datasets, the Recall@20 improved by 9.26%, 3.11% and 2.92%, respectively. Note to Practitioners-The motivation for this article arises from the requirement of efficient recommendation algorithms for e-commerce platforms. Mainstream recommendation algorithms are mainly based on sparse user item history interaction data for recommendation, which receives the limitation of data sparsity problem to a larger extent. Secondly, it is still limited to using the same weights to fuse the features of different modalities in the multimodal recommendation process. In this paper, we propose MMSLRec, which solves the data sparsity problem to a certain extent by using adversarial generative networks and fuse the features from different users using the user co-occurrence matrix. This method achieves better results in different applications. |
|---|---|
| AbstractList | With the rise of e-commerce, personalized recommendation algorithms have received much attention in recent years. Meanwhile, multimodal recommendation algorithms have become the next competitive track of personalized recommendation. The current mainstream recommendation algorithms (e.g., NeuMF, NGCF) rely too much on empirical data and do not effectively utilize multimodal information to model user preferences, while there is the problem of sparsity of user preference data. Secondly, they are still limited to using the same weights to fuse the item features of different modalities, and have the problem of inaccurate recommendation. For this reason, this paper proposes Multi-Modal Self-Supervised Learning Algorithm Based Product Recommendation (MMSLRec). 1) Data augmentation by self-supervised adversarial perturbation generation learning is performed to supplement the missing interaction information of tailed products to solve the common data sparsity problem in product recommendation. 2) A multimodal representation learning module is introduced to capture deep features in image and text modalities using AlexNet and Bert, respectively, in order to avoid the recommendation accuracy degradation problem caused by insufficient data feature learning. 3) Learning the user's perceived strength in different modalities, using the user co-occurrence matrix to fuse the features and make recommendations. Experiments on Amazon-Electronics, Amazon-Clothing, and Amazon-Baby, with different degrees of data sparsity verify the effectiveness of our MMSLRec in solving the data sparsity problem and the multimodal user preference modeling problem, and compared to the state-of-the-art baseline recommendation model on the three datasets, the Recall@20 improved by 9.26%, 3.11% and 2.92%, respectively. Note to Practitioners-The motivation for this article arises from the requirement of efficient recommendation algorithms for e-commerce platforms. Mainstream recommendation algorithms are mainly based on sparse user item history interaction data for recommendation, which receives the limitation of data sparsity problem to a larger extent. Secondly, it is still limited to using the same weights to fuse the features of different modalities in the multimodal recommendation process. In this paper, we propose MMSLRec, which solves the data sparsity problem to a certain extent by using adversarial generative networks and fuse the features from different users using the user co-occurrence matrix. This method achieves better results in different applications. |
| Author | Yang, Heyu Fan, Haiping Gao, Li Chen, Qingkui |
| Author_xml | – sequence: 1 givenname: Li orcidid: 0000-0002-5762-184X surname: Gao fullname: Gao, Li email: gaoli@usst.edu.cn organization: School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, China – sequence: 2 givenname: Haiping surname: Fan fullname: Fan, Haiping organization: School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, China – sequence: 3 givenname: Qingkui surname: Chen fullname: Chen, Qingkui organization: School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, China – sequence: 4 givenname: Heyu surname: Yang fullname: Yang, Heyu organization: School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, China |
| BookMark | eNpNkMtKAzEYhYNUsK0-gOBiXiA1l_knybKWesEWxdb1kCb_1MjMRDJTwbfXoV24OgfOZfFNyKiNLRJyzdmMc2Zut_PNciaYgJmEQgtdnJExB9BUKi1Hg8-BggG4IJOu-2RM5NqwMXleH-o-0HX0ts42WFd0c_jC9B069NkKbWpDu8_m9T6m0H809M4OwWuK_uD67A1dbBpsve1DbC_JeWXrDq9OOiXv98vt4pGuXh6eFvMVdUKwnlouK68LsBI1VmC0Ry1V5VVhdswJbXMHuUahgSvPFENlYecVoC88N87KKeHHX5di1yWsyq8UGpt-Ss7KgUY50CgHGuWJxt_m5rgJiPivb4zMOZO_7uZedQ |
| CODEN | ITASC7 |
| Cites_doi | 10.1109/ICDM.2019.00087 10.1145/3485447.3512104 10.1109/CVPR.2009.5206848 10.1016/j.compag.2020.105779 10.1109/ICoICT61617.2024.10698392 10.1145/3038912.3052569 10.1145/3477495.3532058 10.1145/3394171.3413556 10.1007/s10462-019-09744-1 10.1109/ACCESS.2020.3019464 10.1109/ICCC59590.2023.10507494 10.1109/ICASSP.2018.8462624 10.1109/TMM.2022.3187556 10.1145/2783258.2783273 10.1145/3308558.3313513 10.1007/978-3-030-01240-3_45 10.1145/3477495.3532027 10.1016/j.ins.2019.10.038 10.1109/TNNLS.2022.3154345 10.1145/3343031.3351034 10.1109/ACCESS.2024.3368868 10.1145/3404835.3462862 10.1145/3331184.3331267 10.1016/j.knosys.2019.06.019 10.1049/trit.2020.0031 10.1145/3469213.3470423 10.1109/TMM.2021.3111487 10.1145/3394171.3413653 10.1145/1864708.1864770 10.1145/3209978.3209981 10.1145/3596250 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TASE.2025.3568286 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-3783 |
| EndPage | 15380 |
| ExternalDocumentID | 10_1109_TASE_2025_3568286 10993410 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Major Policy Advisory Research Project of the Shanghai Municipal People’s Government grantid: 2025-A-25 – fundername: National Social Sciences Foundation of China grantid: 22BTQ021 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 O9- OCL PQQKQ RIA RIE RNS AAYXX CITATION |
| ID | FETCH-LOGICAL-c220t-a13fd865a3e8ef598de837fd769b0c28a4c548e28517d070e7a5bd75ed6d19ca3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001494111500010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1545-5955 |
| IngestDate | Sat Nov 29 07:53:52 EST 2025 Wed Aug 27 01:37:50 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c220t-a13fd865a3e8ef598de837fd769b0c28a4c548e28517d070e7a5bd75ed6d19ca3 |
| ORCID | 0000-0002-5762-184X |
| PageCount | 10 |
| ParticipantIDs | crossref_primary_10_1109_TASE_2025_3568286 ieee_primary_10993410 |
| PublicationCentury | 2000 |
| PublicationDate | 20250000 2025-00-00 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 20250000 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on automation science and engineering |
| PublicationTitleAbbrev | TASE |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref35 ref12 ref34 ref15 ref14 Devlin (ref28) 2018 ref36 Van Meteren (ref5) ref11 ref33 ref10 ref32 ref2 Vincent (ref17) 2010; 11 ref1 ref39 ref16 ref38 ref19 ref18 Krizhevsky (ref27) ref24 ref23 ref26 Gulrajani (ref31) ref25 ref20 Ma (ref37) ref22 Glorot (ref30) ref21 ref29 ref8 ref7 ref4 Khosla (ref9); 33 ref3 ref6 |
| References_xml | – ident: ref2 doi: 10.1109/ICDM.2019.00087 – ident: ref10 doi: 10.1145/3485447.3512104 – ident: ref33 doi: 10.1109/CVPR.2009.5206848 – ident: ref3 doi: 10.1016/j.compag.2020.105779 – ident: ref6 doi: 10.1109/ICoICT61617.2024.10698392 – ident: ref34 doi: 10.1145/3038912.3052569 – ident: ref26 doi: 10.1145/3477495.3532058 – ident: ref36 doi: 10.1145/3394171.3413556 – ident: ref8 doi: 10.1007/s10462-019-09744-1 – ident: ref20 doi: 10.1109/ACCESS.2020.3019464 – ident: ref12 doi: 10.1109/ICCC59590.2023.10507494 – start-page: 1 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref37 article-title: Learning disentangled representations for recommendation – ident: ref32 doi: 10.1109/ICASSP.2018.8462624 – ident: ref38 doi: 10.1109/TMM.2022.3187556 – start-page: 47 volume-title: Proc. Mach. Learn. New Inf. Age, MLnet/ECML2000 Workshop ident: ref5 article-title: Using content-based filtering for recommendation – ident: ref16 doi: 10.1145/2783258.2783273 – volume: 11 start-page: 3371 issue: 12 year: 2010 ident: ref17 article-title: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion publication-title: J. Mach. Learn. Res. – ident: ref11 doi: 10.1145/3308558.3313513 – ident: ref29 doi: 10.1007/978-3-030-01240-3_45 – ident: ref39 doi: 10.1145/3477495.3532027 – ident: ref15 doi: 10.1016/j.ins.2019.10.038 – ident: ref22 doi: 10.1109/TNNLS.2022.3154345 – volume: 33 start-page: 18661 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref9 article-title: Supervised contrastive learning – ident: ref13 doi: 10.1145/3343031.3351034 – ident: ref7 doi: 10.1109/ACCESS.2024.3368868 – ident: ref25 doi: 10.1145/3404835.3462862 – ident: ref35 doi: 10.1145/3331184.3331267 – year: 2018 ident: ref28 article-title: BERT: Pre-training of deep bidirectional transformers for language understanding publication-title: arXiv:1810.04805 – start-page: 249 volume-title: Proc. 13th Int. Conf. Artif. Intell. Statist. ident: ref30 article-title: Understanding the difficulty of training deep feedforward neural networks – ident: ref21 doi: 10.1016/j.knosys.2019.06.019 – ident: ref19 doi: 10.1049/trit.2020.0031 – ident: ref18 doi: 10.1145/3469213.3470423 – start-page: 1 volume-title: Proc. Advances Neural Inf. Process. Syst. ident: ref27 article-title: Imagenet classification with deep convolutional neural networks – ident: ref24 doi: 10.1109/TMM.2021.3111487 – ident: ref23 doi: 10.1145/3394171.3413653 – ident: ref14 doi: 10.1145/1864708.1864770 – start-page: 1 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref31 article-title: Improved training of Waserstein gans – ident: ref1 doi: 10.1145/3209978.3209981 – ident: ref4 doi: 10.1145/3596250 |
| SSID | ssj0024890 |
| Score | 2.3891222 |
| Snippet | With the rise of e-commerce, personalized recommendation algorithms have received much attention in recent years. Meanwhile, multimodal recommendation... |
| SourceID | crossref ieee |
| SourceType | Index Database Publisher |
| StartPage | 15371 |
| SubjectTerms | artificial intelligence Collaborative filtering Computational modeling Data mining Data models Feature extraction Motion pictures Prediction algorithms predictive models Recommender system Recommender systems Self-supervised learning Training |
| Title | Multi-Modal Self-Supervised Learning Algorithm-Based Product Recommendation |
| URI | https://ieeexplore.ieee.org/document/10993410 |
| Volume | 22 |
| WOSCitedRecordID | wos001494111500010&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-3783 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0024890 issn: 1545-5955 databaseCode: RIE dateStart: 20040101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46POjBnxPnL3rwJGTrmqZJjlMcgjoGm7JbSZOXOdjWsXX-_SZpxV48eCttQsIXkrzvvX7vIXSnMsVjHRtruRmCYwCGM5rFmAuAUEKoYh8x_XhlgwGfTMSwEqt7LQwA-J_PoO0efSxf52rrXGUdF8Wxp65l6LuMJaVY6zexHvcOFWcSYCoorUKYtk9n3Bs9WSoY0TahSambrl1Ctaoq_lLpH_1zOsfosLIeg1653CdoB5an6KCWU_AMvXhJLX7LtW04grnBo-3KHQgb0EGVTXUa9ObTfD0rPhf4QboPwzLxa-DY6MIOW1ZaaqL3_tP48RlXFROwiqKwwLJLjOYJlQQ4GCq4BktAjWaJyEIVcRkry1AgsmYW03azA5M004yCTnRXKEnOUWOZL-ECBYwbu9uB0K4icSiNDLUyQhKmM5FQRlro_gfCdFUmxkg9oQhF6vBOHd5phXcLNR18tYYlcpd_vL9C-6576eq4Ro1ivYUbtKe-itlmfevX_Ru5YqxW |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwELVQQQIOrEXs5MAJyW0a27F9LKgIRFtVakG9RY49LpWgRSXl-7GdIHrhwC1KnEXP27yZzBuErnWuBTXUOsvNEkwBOM5ZTrGQALGCWNMQMX3p8n5fjMdyUCWrh1wYAAg_n0HDH4ZYvpnrpXeVNX0Ux626jqGvM0qTuEzX-pXWE8Gl4o0CzCRjVRDT3dUctYcdRwYT1iAsLTOnV7ahlboqYVu53_3nB-2hncp-jNplh--jNZgdoO0VVcFD9BSSanFvblzDIbxZPFx--CXhE0xU6alOovbbZL6YFq_v-Fb5C4NS-jXyfPTdvbastVRHz_ed0d0DrmomYJ0kcYFVi1gjUqYICLBMCgOOglrDU5nHOhGKasdRIHGGFjduugNXLDecgUlNS2pFjlBtNp_BMYq4sG6-A2EtTWisrIqNtlIRbnKZMk5O0M0PhNlHKY2RBUoRy8zjnXm8swrvE1T38K00LJE7_eP8Fdp8GPW6Wfex_3SGtvyjSsfHOaoViyVcoA39VUw_F5dhDHwD4PuvnQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multi-Modal+Self-Supervised+Learning+Algorithm-Based+Product+Recommendation&rft.jtitle=IEEE+transactions+on+automation+science+and+engineering&rft.au=Gao%2C+Li&rft.au=Fan%2C+Haiping&rft.au=Chen%2C+Qingkui&rft.au=Yang%2C+Heyu&rft.date=2025&rft.issn=1545-5955&rft.eissn=1558-3783&rft.volume=22&rft.spage=15371&rft.epage=15380&rft_id=info:doi/10.1109%2FTASE.2025.3568286&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TASE_2025_3568286 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1545-5955&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1545-5955&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1545-5955&client=summon |