Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis
We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions-the agents' sum-utility-plus a nonsmooth (extended-valued) convex one. We propose a general unified algorithmic framework for such a class of problems and provide a convergence...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on signal processing Jg. 69; S. 3555 - 3570 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
IEEE
2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 1053-587X, 1941-0476 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions-the agents' sum-utility-plus a nonsmooth (extended-valued) convex one. We propose a general unified algorithmic framework for such a class of problems and provide a convergence analysis leveraging the theory of operator splitting. Distinguishing features of our scheme are: (i) When each of the agent's functions is strongly convex, the algorithm converges at a linear rate, whose dependence on the agents' functions and network topology is decoupled ; (ii) When the objective function is convex (but not strongly convex), similar decoupling as in (i) is established for the coefficient of the proved sublinear rate. This also reveals the role of function heterogeneity on the convergence rate. (iii) The algorithm can adjust the ratio between the number of communications and computations to achieve a rate (in terms of computations) independent on the network connectivity; and (iv) A by-product of our analysis is a tuning recommendation for several existing (non-accelerated) distributed algorithms yielding provably faster (worst-case) convergence rate for the class of problems under consideration. |
|---|---|
| AbstractList | We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions-the agents' sum-utility-plus a nonsmooth (extended-valued) convex one. We propose a general unified algorithmic framework for such a class of problems and provide a convergence analysis leveraging the theory of operator splitting. Distinguishing features of our scheme are: (i) When each of the agent's functions is strongly convex, the algorithm converges at a linear rate, whose dependence on the agents' functions and network topology is decoupled ; (ii) When the objective function is convex (but not strongly convex), similar decoupling as in (i) is established for the coefficient of the proved sublinear rate. This also reveals the role of function heterogeneity on the convergence rate. (iii) The algorithm can adjust the ratio between the number of communications and computations to achieve a rate (in terms of computations) independent on the network connectivity; and (iv) A by-product of our analysis is a tuning recommendation for several existing (non-accelerated) distributed algorithms yielding provably faster (worst-case) convergence rate for the class of problems under consideration. |
| Author | Sun, Ying Scutari, Gesualdo Xu, Jinming Tian, Ye |
| Author_xml | – sequence: 1 givenname: Jinming surname: Xu fullname: Xu, Jinming email: jimmyxu@zju.edu.cn organization: College of Control Science and Engineering, Zhejiang University, Hangzhou, China – sequence: 2 givenname: Ye orcidid: 0000-0001-9085-6280 surname: Tian fullname: Tian, Ye email: tian110@purdue.edu organization: School of Industrial Engineering, Purdue University, West-Lafayette, IN, USA – sequence: 3 givenname: Ying orcidid: 0000-0002-9709-6509 surname: Sun fullname: Sun, Ying email: sun578@purdue.edu organization: School of Electrical Engineering and Computer Science, The Pennsylvania State University, State College, PA, USA – sequence: 4 givenname: Gesualdo orcidid: 0000-0002-6453-6870 surname: Scutari fullname: Scutari, Gesualdo email: gscutari@purdue.edu organization: School of Industrial Engineering, Purdue University, West-Lafayette, IN, USA |
| BookMark | eNp9kEFLw0AQhRepYFu9C14CnlN3ks2m661Uq0Khgi14kbBJJnVrkq27W6X-ere2ePDgZWaY-d4w83qk0-oWCTkHOgCg4mr-9DiIaASDmA55kooj0gXBIKQs5R1f0yQOk2H6fEJ61q4oBcYE75KXG2WdUfnGYRmM6qU2yr02Nqi0Cca6WWurHAaztVON-pJO6fY6WLSqUh6fGNngpzZvgWxLT7cfaJbYFhiMWllvrbKn5LiStcWzQ-6TxeR2Pr4Pp7O7h_FoGhZxkroQgZdQ0FIkIgeQVV5VnEMuuYgSTKXwXeGBFACQ5QwYhQpzjkzGRVpwGvfJ5X7v2uj3DVqXrfTG-CNsFiUJxDzywVN8TxVGW2uwygrlfn5yRqo6A5rtrMy8ldnOyuxgpRfSP8K1UY002_8kF3uJQsRfXDDmZyL-BhtXgj4 |
| CODEN | ITPRED |
| CitedBy_id | crossref_primary_10_1109_TCNS_2024_3393668 crossref_primary_10_1002_sta4_70064 crossref_primary_10_1109_TAC_2023_3279901 crossref_primary_10_1109_TSP_2022_3184770 crossref_primary_10_1007_s11590_023_02011_x crossref_primary_10_1109_TSP_2024_3514676 crossref_primary_10_1109_TAC_2021_3122586 crossref_primary_10_1109_TAC_2024_3524119 crossref_primary_10_1007_s10957_024_02393_7 crossref_primary_10_1109_OJSP_2025_3557332 crossref_primary_10_1109_TAC_2025_3552743 crossref_primary_10_1080_10618600_2024_2431060 crossref_primary_10_1109_TAC_2024_3471854 crossref_primary_10_1007_s10957_022_02061_8 crossref_primary_10_1109_TIT_2023_3267742 crossref_primary_10_1016_j_sigpro_2025_110275 crossref_primary_10_1007_s10957_024_02554_8 crossref_primary_10_1109_TAC_2023_3301289 crossref_primary_10_1109_TNNLS_2022_3208086 crossref_primary_10_1007_s10994_024_06537_8 crossref_primary_10_1109_TSMC_2024_3405453 crossref_primary_10_1109_TSP_2023_3250839 crossref_primary_10_1109_TSIPN_2022_3190743 crossref_primary_10_1109_TSP_2025_3544517 crossref_primary_10_1007_s10107_023_01997_7 crossref_primary_10_1007_s10957_024_02385_7 crossref_primary_10_1109_TAC_2024_3360287 crossref_primary_10_1109_TSIPN_2025_3600766 crossref_primary_10_1109_TSIPN_2022_3203860 crossref_primary_10_1109_TIT_2022_3176253 crossref_primary_10_1109_TNSE_2022_3155287 crossref_primary_10_1007_s10898_022_01221_4 crossref_primary_10_1109_TCE_2024_3506915 crossref_primary_10_1109_TCNS_2024_3354882 crossref_primary_10_1109_TAC_2021_3116116 crossref_primary_10_1109_TAC_2023_3244904 crossref_primary_10_1109_TAC_2023_3327940 crossref_primary_10_1109_TSMC_2025_3565568 crossref_primary_10_3390_e24091278 crossref_primary_10_1016_j_ifacol_2022_07_264 crossref_primary_10_1109_TAC_2022_3173171 crossref_primary_10_1016_j_neunet_2024_106325 crossref_primary_10_1038_s41598_025_90005_3 crossref_primary_10_1109_TSIPN_2023_3290397 |
| Cites_doi | 10.1109/CDC40024.2019.9029902 10.1109/TAC.2017.2730481 10.1109/TSP.2018.2875883 10.1007/s10107-018-01357-w 10.1109/TCNS.2017.2698261 10.1109/ALLERTON.2017.8262874 10.1109/TAC.2017.2713046 10.1109/CDC.2012.6425904 10.1109/CAMSAP45676.2019.9022451 10.1137/16M1084316 10.1109/TSIPN.2018.2846183 10.1016/j.sysconle.2004.02.022 10.1109/TAC.2018.2805260 10.1016/j.ifacol.2019.12.181 10.1109/CDC.2015.7402509 10.23919/ACC.2019.8814838 10.1109/TAC.2020.2972824 10.1017/CBO9780511840371 10.1109/TSP.2015.2461520 10.1007/978-1-4419-8853-9 10.1137/14096668X 10.1109/TAC.2020.3009363 10.1109/TSP.2019.2926022 10.1109/LCSYS.2018.2834316 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TSP.2021.3086579 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Xplore CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1941-0476 |
| EndPage | 3570 |
| ExternalDocumentID | 10_1109_TSP_2021_3086579 9447939 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Science Foundation; USA NSF grantid: 1719205 funderid: 10.13039/100000001 – fundername: ARO grantid: W911NF1810238 – fundername: National Natural Science Foundation of China; NSF of China grantid: 62003302; 62088101; 61922058; U1909207 funderid: 10.13039/501100001809 – fundername: Fundamental Research Funds for the Central Universities grantid: 2020QNA5014 funderid: 10.13039/501100012226 – fundername: ONR |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 53G 5GY 5VS 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AJQPL AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 E.L EBS EJD F5P HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c357t-e16d1c0d959b11afbff661ba6925e7a99b19d1c7111e4b41401feb6e4a3c7c603 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 48 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000673500300001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1053-587X |
| IngestDate | Mon Jun 30 10:22:51 EDT 2025 Tue Nov 18 22:21:57 EST 2025 Sat Nov 29 04:10:53 EST 2025 Wed Aug 27 02:23:35 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c357t-e16d1c0d959b11afbff661ba6925e7a99b19d1c7111e4b41401feb6e4a3c7c603 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-9085-6280 0000-0002-6453-6870 0000-0002-9709-6509 |
| PQID | 2551362513 |
| PQPubID | 85478 |
| PageCount | 16 |
| ParticipantIDs | proquest_journals_2551362513 crossref_citationtrail_10_1109_TSP_2021_3086579 ieee_primary_9447939 crossref_primary_10_1109_TSP_2021_3086579 |
| PublicationCentury | 2000 |
| PublicationDate | 20210000 2021-00-00 20210101 |
| PublicationDateYYYYMMDD | 2021-01-01 |
| PublicationDate_xml | – year: 2021 text: 20210000 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on signal processing |
| PublicationTitleAbbrev | TSP |
| PublicationYear | 2021 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref12 p (ref6) 2016; 2 ref15 ref14 ref31 xu (ref25) 0 ref11 ref32 ref10 ref1 ref16 scaman (ref13) 0 ref19 ref18 xu (ref28) 2020 alghunaim (ref17) 2019 nedi? (ref5) 0 asuncion (ref36) 2007 xu (ref2) 0 ref24 ref23 ref26 ref20 ref22 ref21 alghunaim (ref27) 2021; 66 agarwal (ref34) 0 ref29 ref8 ref7 auzinger (ref30) 2011 ref4 ref3 nesterov (ref37) 2004 shamir (ref33) 0 sun (ref9) 2019 |
| References_xml | – ident: ref15 doi: 10.1109/CDC40024.2019.9029902 – ident: ref31 doi: 10.1109/TAC.2017.2730481 – start-page: 1000 year: 0 ident: ref33 article-title: Communication-efficient distributed optimization using an approximate newton-type method publication-title: Proc Int Conf Mach Learn – year: 2019 ident: ref17 article-title: A linearly convergent proximal gradient algorithm for decentralized optimization publication-title: Proc Adv Neural Inf Process Syst – volume: 2 start-page: 120 year: 2016 ident: ref6 article-title: Next: In-network nonconvex optimization publication-title: IEEE Trans Signal Inf Process Netw – ident: ref29 doi: 10.1109/TSP.2018.2875883 – ident: ref8 doi: 10.1007/s10107-018-01357-w – start-page: 3027 year: 0 ident: ref13 article-title: Optimal algorithms for smooth and strongly convex distributed optimization in networks publication-title: Proc 34th Int Conf Mach Learn – ident: ref7 doi: 10.1109/TCNS.2017.2698261 – ident: ref24 doi: 10.1109/ALLERTON.2017.8262874 – start-page: 2381 year: 0 ident: ref25 article-title: Accelerated primal-dual algorithms for distributed smooth convex optimization over networks publication-title: Proc 23rd Int Conf Artif Intell Statist – ident: ref20 doi: 10.1109/TAC.2017.2713046 – ident: ref16 doi: 10.1109/CDC.2012.6425904 – ident: ref1 doi: 10.1109/CAMSAP45676.2019.9022451 – ident: ref10 doi: 10.1137/16M1084316 – year: 2020 ident: ref28 article-title: Distributed algorithms for composite optimization: unified framework and convergence analysis – start-page: 2309 year: 0 ident: ref2 article-title: A unified algorithmic framework for distributed composite optimization publication-title: Proc 59th IEEE Conf Decis Control – ident: ref14 doi: 10.1109/TSIPN.2018.2846183 – ident: ref35 doi: 10.1016/j.sysconle.2004.02.022 – ident: ref19 doi: 10.1109/TAC.2018.2805260 – start-page: 3950 year: 0 ident: ref5 article-title: Geometrically convergent distributed optimization with uncoordinated step-sizes publication-title: Proc Amer Control Conf – year: 2007 ident: ref36 article-title: UCI machine learning repository – ident: ref26 doi: 10.1016/j.ifacol.2019.12.181 – ident: ref4 doi: 10.1109/CDC.2015.7402509 – year: 2011 ident: ref30 article-title: Iterative solution of large linear systems publication-title: Lecture notes – start-page: 37 year: 0 ident: ref34 article-title: Fast global convergence rates of gradient methods for high-dimensional statistical recovery publication-title: Proc Adv Neural Inf Process Syst – ident: ref23 doi: 10.23919/ACC.2019.8814838 – ident: ref21 doi: 10.1109/TAC.2020.2972824 – ident: ref32 doi: 10.1017/CBO9780511840371 – ident: ref18 doi: 10.1109/TSP.2015.2461520 – year: 2004 ident: ref37 publication-title: Introductory Lectures on Convex Optimization A Basic Course doi: 10.1007/978-1-4419-8853-9 – ident: ref3 doi: 10.1137/14096668X – ident: ref12 doi: 10.1109/TSP.2018.2875883 – volume: 66 year: 2021 ident: ref27 article-title: Decentralized proximal gradient algorithms with linear convergence rates publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.2020.3009363 – ident: ref11 doi: 10.1109/TSP.2019.2926022 – year: 2019 ident: ref9 article-title: Distributed optimization based on gradient-tracking revisited: Enhancing convergence rate via surrogation – ident: ref22 doi: 10.1109/LCSYS.2018.2834316 |
| SSID | ssj0014496 |
| Score | 2.6049955 |
| Snippet | We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions-the agents' sum-utility-plus a nonsmooth... We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions–the agents’ sum-utility–plus a nonsmooth... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 3555 |
| SubjectTerms | Algorithms Convergence Decoupling Distributed algorithms Distributed optimization Electronic mail Heterogeneity linear convergence Network topologies Network topology Operators (mathematics) Optimization Signal processing algorithms Sun Tuning |
| Title | Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis |
| URI | https://ieeexplore.ieee.org/document/9447939 https://www.proquest.com/docview/2551362513 |
| Volume | 69 |
| WOSCitedRecordID | wos000673500300001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0476 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014496 issn: 1053-587X databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5q8aAHX1WsVtmDF8G0SbPJZr0VtXiqBRV6kbDZzGqhTaWv3-9sHkVQBC8hhNkQdjaz3-w8PoArH5UhTyNyVBKlDg_ooqQxDipOIKmbaC_SOdmEGAyi0UgOa3CzqYVBxDz5DNv2No_lpzO9skdlHcntOZDcgi0hwqJWaxMx4Dzn4iK44DtBJEZVSNKVnZfnITmCXa_tE34PbNLWty0o51T5YYjz3aW__7_vOoC9EkWyXqH2Q6hhdgS733oLNuDt3rbEtWxWmLLe5H02Hy8_pgtGIJVZI2CTtZA9kcWYlqWYt4wAqCFIyvpVxhZTWUrS2bqo0URW9TA5htf-w8vdo1NyKTjaD8TSQS9MPe2mMpCJ5ymTGEM7c6JC2Q1QKElPJQkIMn3IE27dLoNJiFz5WujQ9U-gns0yPAXGtYgC7etEu5qTCVAhCUgpjLJBRSWa0KmmN9Zlo3HLdzGJc4fDlTEpJLYKiUuFNOF6M-KzaLLxh2zDKmAjV859E1qVBuPyL1zEXcteQw6e55_9Puocduy7iyOVFtSX8xVewLZeL8eL-WW-wL4AFlTPwg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1ZS8QwEB68QH3wFtczD74I1u2RNo1voi6KugqusC9S0nSignZlr9_vpMciKIIvpZQJLZl08k3m-AAOA1SGPI3YUWmcOTyki5LGOKg4gSQ_1V6sC7IJ0W7H3a58mILjSS0MIhbJZ3hib4tYftbTI3tU1pTcngPJaZgNOffdslprEjPgvGDjIsAQOGEsunVQ0pXNzuMDuYK-dxIQgg9t2ta3TahgVflhiov9pbX8vy9bgaUKR7KzUvGrMIX5Gix-6y64Ds8Xtimu5bPCjJ29v_T6b8PXjwEjmMqsGbDpWsjuyWZ8VMWYp4wgqCFQylp1zhZTeUbS-bis0kRWdzHZgKfWZef8yqnYFBwdhGLooBdlnnYzGcrU85RJjaG9OVWR9EMUStJTSQKCjB_ylFvHy2AaIVeBFjpyg02YyXs5bgHjWsShDnSqXc3JCKiIBKQURtmwohINaNbTm-iq1bhlvHhPCpfDlQkpJLEKSSqFNOBoMuKzbLPxh-y6VcBErpr7BuzWGkyq_3CQ-Ja_hlw8L9j-fdQBzF917m6T2-v2zQ4s2PeUByy7MDPsj3AP5vR4-Dbo7xeL7QsNbtMJ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Distributed+Algorithms+for+Composite+Optimization%3A+Unified+Framework+and+Convergence+Analysis&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Xu%2C+Jinming&rft.au=Tian%2C+Ye&rft.au=Sun%2C+Ying&rft.au=Scutari%2C+Gesualdo&rft.date=2021&rft.issn=1053-587X&rft.eissn=1941-0476&rft.volume=69&rft.spage=3555&rft.epage=3570&rft_id=info:doi/10.1109%2FTSP.2021.3086579&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TSP_2021_3086579 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon |