Decentralized Rank-Adaptive Matrix Factorization-Part I: Algorithm Development

Factorizing a low-rank matrix into two matrix factors with low dimensions from its noisy observations is a classical but challenging problem arising from real-world applications. This paper develops decentralized matrix factorization algorithms, i.e., factorizing a matrix whose columns are stored di...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on signal processing Jg. 73; S. 4124 - 4140
Hauptverfasser:	Jiao, Yuchen, Gu, Yuantao, Chang, Tsung-Hui, Luo, Zhi-Quan
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Accuracy Adaptive algorithms Algorithms Convergence decentralized Factorization Image reconstruction Linear programming Low rank Matrix decomposition matrix factorization Noise Principal component analysis rank adaptive Regularization Reviews Signal processing algorithms Sparse matrices
ISSN:	1053-587X, 1941-0476
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	Factorizing a low-rank matrix into two matrix factors with low dimensions from its noisy observations is a classical but challenging problem arising from real-world applications. This paper develops decentralized matrix factorization algorithms, i.e., factorizing a matrix whose columns are stored distributively over a network without a central agent. The performance of existing algorithms relies heavily on the accuracy of the matrix rank estimate. However, acquiring an accurate estimate is difficult in the distributed setting. In this paper and its Part II, we address this problem by introducing a novel regularization into the objective function to induce the solution with correct rank. Based on this, we propose a rank-adaptive decentralized MF algorithm. In Part I, we delineate the algorithm development from the centralized with known rank, decentralized with known rank, to the rank-adaptive decentralized settings. For the centralized algorithm, we present the first globally linear convergence analysis for the alternating gradient descent method. In the Part II, we analyze conditions for which the proposed rank-adaptive decentralized MF algorithm converges to the global solution with the correct rank. Numerical experiments based on both synthetic and real-world datasets are presented in this paper to demonstrate the effectiveness of the proposed algorithms and corroborate the theoretical claims.
AbstractList	Factorizing a low-rank matrix into two matrix factors with low dimensions from its noisy observations is a classical but challenging problem arising from real-world applications. This paper develops decentralized matrix factorization algorithms, i.e., factorizing a matrix whose columns are stored distributively over a network without a central agent. The performance of existing algorithms relies heavily on the accuracy of the matrix rank estimate. However, acquiring an accurate estimate is difficult in the distributed setting. In this paper and its Part II, we address this problem by introducing a novel regularization into the objective function to induce the solution with correct rank. Based on this, we propose a rank-adaptive decentralized MF algorithm. In Part I, we delineate the algorithm development from the centralized with known rank, decentralized with known rank, to the rank-adaptive decentralized settings. For the centralized algorithm, we present the first globally linear convergence analysis for the alternating gradient descent method. In the Part II, we analyze conditions for which the proposed rank-adaptive decentralized MF algorithm converges to the global solution with the correct rank. Numerical experiments based on both synthetic and real-world datasets are presented in this paper to demonstrate the effectiveness of the proposed algorithms and corroborate the theoretical claims.
Author	Luo, Zhi-Quan Chang, Tsung-Hui Jiao, Yuchen Gu, Yuantao
Author_xml	– sequence: 1 givenname: Yuchen orcidid: 0009-0007-4371-3341 surname: Jiao fullname: Jiao, Yuchen email: jiaoyc201108@163.com organization: School of Information and Electronics, Beijing Institute of Technology, Beijing, China – sequence: 2 givenname: Yuantao orcidid: 0000-0002-8427-1021 surname: Gu fullname: Gu, Yuantao email: gyt@tsinghua.edu.cn organization: Department of Electronic Engineering, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China – sequence: 3 givenname: Tsung-Hui orcidid: 0000-0003-1349-2764 surname: Chang fullname: Chang, Tsung-Hui email: tsunghui.chang@ieee.org organization: The Chinese University of Hong Kong (CUHK), Shenzhen, China – sequence: 4 givenname: Zhi-Quan orcidid: 0000-0003-3995-914X surname: Luo fullname: Luo, Zhi-Quan email: luozq@cuhk.edu.cn organization: The Chinese University of Hong Kong (CUHK), Shenzhen, China
BookMark	eNpNkE1PwkAQhjcGEwG9e_DQxHNx9rv1RkCUBJUoJt4223aqxdLidiHKr7cEDp5mMnned5KnRzpVXSEhlxQGlEJ8s3idDxgwMeBCSYD4hHRpLGgIQqtOu4PkoYz0-xnpNc0SgAoRqy55GmOKlXe2LHaYBS-2-gqHmV37YovBo_Wu-AkmNvW1K3bWF3UVzq3zwfQ2GJYf7dF_roIxbrGs16u255yc5rZs8OI4--RtcrcYPYSz5_vpaDgLUyakDxOW8EgrpCDySDMOUcaSKFUyVllOFc1jTXkskEUUM51m2iaQQcK4VhpQSt4n14fetau_N9h4s6w3rmpfGs40pVpTyVoKDlTq6qZxmJu1K1bW_RoKZm_NtNbM3po5WmsjV4dIgYj_cA1SMMb_AB7caYo
CODEN	ITPRED
Cites_doi	10.1145/3584862 10.1109/TSP.2008.917356 10.1007/s10589-021-00328-w 10.1561/2400000003 10.1109/TNNLS.2018.2851957 10.1093/imaiai/iay003 10.3390/app122111118 10.1109/TNNLS.2016.2597444 10.1109/tsp.2022.3151505 10.1109/TASSP.1985.1164557 10.1016/j.patcog.2023.109637 10.1109/TSP.2013.2279080 10.1109/TIT.2016.2598574 10.1007/s10107-018-1285-1 10.1109/5.726791 10.1145/2488608.2488693 10.1109/ICACC.2015.82 10.3934/ipi.2015.9.601 10.1093/imanum/drz061 10.1109/ICASSP.2002.5743991 10.1109/TIP.2017.2762595 10.1109/ICASSP.2017.7952998 10.1137/S0036144503423264 10.1109/TSP.2018.2870353 10.1109/ICASSP.2012.6288528 10.1109/TSP.2024.3465049 10.1109/TIT.2021.3049171 10.1016/0024-3795(92)90407-2 10.1137/14096668X 10.1016/j.neucom.2012.12.055 10.1109/SAMPTA.2015.7148925 10.1109/TSP.2015.2472372 10.1109/LSP.2020.2988596 10.1109/TSP.2012.2232661 10.1017/cbo9780511794308.006 10.1109/TNNLS.2017.2766160 10.1016/0169-7439(87)80084-9 10.1109/TrustCom60117.2023.00148 10.21037/atm.2017.07.12 10.1109/TIT.2010.2046205 10.1109/TBDATA.2022.3163584 10.1109/ICASSP.2016.7472577
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TSP.2024.3465009
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1941-0476
EndPage	4140
ExternalDocumentID	10_1109_TSP_2024_3465009 10705422
Genre	orig-research
GrantInformation_xml	– fundername: Shenzhen Science and Technology Program grantid: ZDSYS20230626091302006; RCJC20210609104448114 – fundername: NSAF grantid: U2230201 – fundername: Guangdong Provincial Key Laboratory of Big Data Computing – fundername: NSFC, China grantid: 62071409; 62301037 funderid: 10.13039/501100001809 – fundername: BNRist Youth Innovation Fund grantid: BNR2024RC01001 – fundername: GuoQiang Institute, Tsinghua University
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 53G 5GY 5VS 6IK 85S 97E AAJGR AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AJQPL AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 E.L EBS EJD F5P HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c245t-b2b3876e104f872308d2b8c6596df161f971394e281ed7cd7ab0d0b237670e553
IEDL.DBID	RIE
ISICitedReferencesCount	1
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001615304500003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1053-587X
IngestDate	Thu Nov 13 06:21:57 EST 2025 Sat Nov 29 06:45:42 EST 2025 Wed Nov 19 08:27:18 EST 2025
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c245t-b2b3876e104f872308d2b8c6596df161f971394e281ed7cd7ab0d0b237670e553
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0009-0007-4371-3341 0000-0003-3995-914X 0000-0002-8427-1021 0000-0003-1349-2764
PQID	3271177152
PQPubID	85478
PageCount	17
ParticipantIDs	proquest_journals_3271177152 crossref_primary_10_1109_TSP_2024_3465009 ieee_primary_10705422
PublicationCentury	2000
PublicationDate	20250000 2025-00-00 20250101
PublicationDateYYYYMMDD	2025-01-01
PublicationDate_xml	– year: 2025 text: 20250000
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on signal processing
PublicationTitleAbbrev	TSP
PublicationYear	2025
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref53 ref52 ref11 ref55 ref10 ref54 ref17 ref16 Li (ref30) 2017 Tu (ref35) 2016 ref19 ref18 Balsubramani (ref22) 2013 ref51 ref50 Ge (ref37) 2016 ref46 ref45 Ge (ref44) 2015 ref47 ref42 ref41 Bhojanapalli (ref39) 2016 ref49 ref8 Zhu (ref7) 2019 ref9 ref4 ref3 ref6 ref5 ref40 Saade (ref24) 2015 ref36 ref31 ref33 ref32 ref1 Zheng (ref34) 2015 Pei (ref2) 2010; 2 Ye (ref48) 2021; 22 Lee (ref43) 2016 ref23 ref26 ref25 ref20 ref21 ref28 ref27 ref29 Ge (ref38) 2017
References_xml	– ident: ref4 doi: 10.1145/3584862 – ident: ref16 doi: 10.1109/TSP.2008.917356 – ident: ref27 doi: 10.1007/s10589-021-00328-w – ident: ref51 doi: 10.1561/2400000003 – ident: ref28 doi: 10.1109/TNNLS.2018.2851957 – ident: ref40 doi: 10.1093/imaiai/iay003 – volume-title: Proc. NeurIPS year: 2016 ident: ref39 article-title: Global optimality of local search for low rank matrix recovery – ident: ref12 doi: 10.3390/app122111118 – ident: ref15 doi: 10.1109/TNNLS.2016.2597444 – ident: ref3 doi: 10.1109/tsp.2022.3151505 – start-page: 1 volume-title: Proc. NeurIPS year: 2015 ident: ref24 article-title: Matrix completion from fewer entries: Spectral detectability and rank estimation – ident: ref18 doi: 10.1109/TASSP.1985.1164557 – ident: ref21 doi: 10.1016/j.patcog.2023.109637 – start-page: 1233 volume-title: Proc. ICML year: 2017 ident: ref38 article-title: No spurious local minima in nonconvex low rank problems: A unified geometric analysis – ident: ref14 doi: 10.1109/TSP.2013.2279080 – ident: ref29 doi: 10.1109/TIT.2016.2598574 – ident: ref50 doi: 10.1007/s10107-018-1285-1 – volume: 22 start-page: 1 issue: 1 year: 2021 ident: ref48 article-title: DeEPCA: Decentralized exact PCA with linear convergence rate publication-title: J. Mach. Learn. Res – ident: ref55 doi: 10.1109/5.726791 – start-page: 1 volume-title: Proc. NeurIPS year: 2016 ident: ref37 article-title: Matrix completion has no spurious local minimum – ident: ref33 doi: 10.1145/2488608.2488693 – start-page: 1 volume-title: Proc. NeurIPS year: 2019 ident: ref7 article-title: Distributed low-rank matrix factorization with exact consensus – ident: ref1 doi: 10.1109/ICACC.2015.82 – year: 2017 ident: ref30 article-title: Geometry of factored nuclear norm regularization – start-page: 1246 volume-title: Proc. COLT year: 2016 ident: ref43 article-title: Gradient descent only converges to minimizers – ident: ref26 doi: 10.3934/ipi.2015.9.601 – ident: ref41 doi: 10.1093/imanum/drz061 – ident: ref45 doi: 10.1109/ICASSP.2002.5743991 – ident: ref25 doi: 10.1109/TIP.2017.2762595 – ident: ref47 doi: 10.1109/ICASSP.2017.7952998 – ident: ref54 doi: 10.1137/S0036144503423264 – ident: ref36 doi: 10.1109/TSP.2018.2870353 – ident: ref13 doi: 10.1109/ICASSP.2012.6288528 – ident: ref19 doi: 10.1109/TSP.2024.3465049 – ident: ref42 doi: 10.1109/TIT.2021.3049171 – ident: ref49 doi: 10.1016/0024-3795(92)90407-2 – start-page: 797 volume-title: Proc. COLT year: 2015 ident: ref44 article-title: Escaping from saddle points: Online stochastic gradient for tensor decomposition – ident: ref53 doi: 10.1137/14096668X – ident: ref8 doi: 10.1016/j.neucom.2012.12.055 – start-page: 964 volume-title: Proc. ICML year: 2016 ident: ref35 article-title: Low-rank solutions of linear matrix equations via procrustes flow – ident: ref32 doi: 10.1109/SAMPTA.2015.7148925 – ident: ref46 doi: 10.1109/TSP.2015.2472372 – ident: ref31 doi: 10.1109/LSP.2020.2988596 – ident: ref17 doi: 10.1109/TSP.2012.2232661 – start-page: 1 volume-title: Proc. NeurIPS year: 2013 ident: ref22 article-title: The fast convergence of incremental PCA – ident: ref52 doi: 10.1017/cbo9780511794308.006 – ident: ref20 doi: 10.1109/TNNLS.2017.2766160 – ident: ref6 doi: 10.1016/0169-7439(87)80084-9 – start-page: 1 volume-title: Proc. NeurIPS year: 2015 ident: ref34 article-title: A convergent gradient descent algorithm for rank minimization and semidefinite programming from random linear measurements – volume: 2 start-page: 281 volume-title: Proc. Int. Conf. Intell. Comput. Technol. Automat. year: 2010 ident: ref2 article-title: Extended semi-supervised matrix factorization for clustering – ident: ref10 doi: 10.1109/TrustCom60117.2023.00148 – ident: ref11 doi: 10.21037/atm.2017.07.12 – ident: ref23 doi: 10.1109/TIT.2010.2046205 – ident: ref5 doi: 10.1109/TBDATA.2022.3163584 – ident: ref9 doi: 10.1109/ICASSP.2016.7472577
SSID	ssj0014496
Score	2.4729888
Snippet	Factorizing a low-rank matrix into two matrix factors with low dimensions from its noisy observations is a classical but challenging problem arising from...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	4124
SubjectTerms	Accuracy Adaptive algorithms Algorithms Convergence decentralized Factorization Image reconstruction Linear programming Low rank Matrix decomposition matrix factorization Noise Principal component analysis rank adaptive Regularization Reviews Signal processing algorithms Sparse matrices
Title	Decentralized Rank-Adaptive Matrix Factorization-Part I: Algorithm Development
URI	https://ieeexplore.ieee.org/document/10705422 https://www.proquest.com/docview/3271177152
Volume	73
WOSCitedRecordID	wos001615304500003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0476 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014496 issn: 1053-587X databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8MwGA46POjBz4nTKT148ZDZJmmTeBvqUNAxdMJupU3e6lC3sQ8Rf71vsk4G4sFbKQ2UJ8mb58n7RcgpAyZtkWQ0jnKgwqAd1CIT1EQhhFoJHRpfZ_ZOttuq19OdMlnd58IAgA8-g4Z79L58OzQzd1WGO1wiw2BocVelTObJWj8uAyF8My7kC5zGSvYWPslQn3cfO6gEmWhwgYTExR4unUG-qcovS-yPl9bWP39sm2yWPDJozid-h6zAYJdsLFUX3CPtKyhDL_tfYIOHbPBKmzYbOQMX3LvS_J9By7fbKXMxaQfXUXB7ETTfnvHl9OU9WIopqpKn1nX38oaW7ROoYSKe0pzlHG0doOAqlESpoSzLlUlindgCiV6hUaBqAUxFYKWxMstDG-YuTEaGEMd8n1QGwwEckEBzBczyxES5FZrz3Ma8EFpYVFMRy1SNnC0ATUfzKhmpVxehThH81IGfluDXSNUBuPTdHLsaqS-mIC330STlTDqvMpKMwz-GHZF15lry-luROqlMxzM4JmvmY9qfjE_8EvkGIQK3_A
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8MwGA6ignrwc-L87MGLh8w0SZfE21CH4hxDJ-xW2uStDnUTnSL-et9knQzEg7dSGihPkjfPk_eLkEMOXLmintEkzoFKi3bQyExSGzNgRkvDbKgz21Lttu71TKdMVg-5MAAQgs-g5h-DL98N7bu_KsMdrpBhcLS4c4mUnI3TtX6cBlKGdlzIGARNtOpNvJLMHHdvO6gFuawJiZTERx9OnUKhrcovWxwOmObKP39tlSyXTDJqjKd-jczAYJ0sTdUX3CDtMyiDL_tf4KKbbPBIGy578SYuuvbF-T-jZmi4U2Zj0g6upOjyJGo83ePL0cNzNBVVVCF3zfPu6QUtGyhQy2UyojnPBVo7QMlVaIViQzuea1tPTN0VSPUKgxLVSOA6BqesU1nOHMt9oIxikCRik8wOhgPYIpERGrgTdRvnThohcpeIQhrpUE_FPNNVcjQBNH0Z18lIg75gJkXwUw9-WoJfJRUP4NR3Y-yqZHcyBWm5k95SwZX3KyPN2P5j2AFZuOhet9LWZftqhyxy36A33JHsktnR6zvskXn7Meq_ve6H5fIN4lK7Qw
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Decentralized+Rank-Adaptive+Matrix+Factorization-Part+I%3A+Algorithm+Development&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Jiao%2C+Yuchen&rft.au=Gu%2C+Yuantao&rft.au=Chang%2C+Tsung-Hui&rft.au=Luo%2C+Zhi-Quan&rft.date=2025&rft.pub=IEEE&rft.issn=1053-587X&rft.volume=73&rft.spage=4124&rft.epage=4140&rft_id=info:doi/10.1109%2FTSP.2024.3465009&rft.externalDocID=10705422
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon