A Distributed, Asynchronous, and Incremental Algorithm for Nonconvex Optimization: An ADMM Approach
The alternating direction method of multipliers (ADMM) has been popular for solving many signal processing problems, convex or nonconvex. In this paper, we study an asynchronous implementation of ADMM for solving a nonconvex nonsmooth optimization problem, whose objective is the sum of a number of c...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on control of network systems Jg. 5; H. 3; S. 935 - 945 |
|---|---|
| 1. Verfasser: | |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Piscataway
IEEE
01.09.2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 2325-5870, 2372-2533 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | The alternating direction method of multipliers (ADMM) has been popular for solving many signal processing problems, convex or nonconvex. In this paper, we study an asynchronous implementation of ADMM for solving a nonconvex nonsmooth optimization problem, whose objective is the sum of a number of component functions. The proposed algorithm allows the problem to be solved in a distributed, asynchronous, and incremental manner. First, the component functions can be distributed to different computing nodes, which perform the updates asynchronously without coordinating with each other. Two sources of asynchrony are covered by our algorithm: One is caused by the heterogeneity of the computational nodes and the other arises from unreliable communication links. Second, the algorithm can be viewed as implementing an incremental algorithm where at each step the (possibly delayed) gradients of only a subset of component functions are updated. We show that when certain bounds are imposed on the level of asynchrony, the proposed algorithm converges to the set of stationary solutions (resp. optimal solutions) for the nonconvex (resp. convex) problem, with a global sublinear rate. |
|---|---|
| AbstractList | The alternating direction method of multipliers (ADMM) has been popular for solving many signal processing problems, convex or nonconvex. In this paper, we study an asynchronous implementation of ADMM for solving a nonconvex nonsmooth optimization problem, whose objective is the sum of a number of component functions. The proposed algorithm allows the problem to be solved in a distributed, asynchronous, and incremental manner. First, the component functions can be distributed to different computing nodes, which perform the updates asynchronously without coordinating with each other. Two sources of asynchrony are covered by our algorithm: One is caused by the heterogeneity of the computational nodes and the other arises from unreliable communication links. Second, the algorithm can be viewed as implementing an incremental algorithm where at each step the (possibly delayed) gradients of only a subset of component functions are updated. We show that when certain bounds are imposed on the level of asynchrony, the proposed algorithm converges to the set of stationary solutions (resp. optimal solutions) for the nonconvex (resp. convex) problem, with a global sublinear rate. |
| Author | Hong, Mingyi |
| Author_xml | – sequence: 1 givenname: Mingyi orcidid: 0000-0003-1263-9365 surname: Hong fullname: Hong, Mingyi email: mhong@umn.edu organization: Department of Industrial and Manufacturing Systems Engineering, Iowa State University, Ames, IA, USA |
| BookMark | eNp9UMlOwzAQtRBIQOkHIC6WuJJie5o44Ra1bBLLgXKOHGdKjVq72C6ifD0uRRw4cJoZzVv03iHZtc4iIcecDThn1flk9PA0EIzLgShyOSzYDjkQIEUmcoDdzS7yLC8l2yf9EF4ZY1zk6YYDoms6NiF6064idme0DmurZ95ZtwpnVNmO3lrtcYE2qjmt5y_Omzhb0Knz9MFZ7ew7ftDHZTQL86micfaC1pbW4_t7Wi-X3ik9OyJ7UzUP2P-ZPfJ8dTkZ3WR3j9e3o_ou06KCmAHrFLSlaCvsCq1lWQFr9bBqUUORBgdgefoDcFRtm6MY6hZhiKLMC2Qd9MjpVjfZvq0wxObVrbxNlo3gXHKZVCCh-BalvQvB47RZerNQft1w1mzqbDZ1Nps6m586E0f-4WgTv9NGr8z8X-bJlmkQ8ddJlikxCPgCA-CEMw |
| CODEN | ITCNAY |
| CitedBy_id | crossref_primary_10_1109_TSP_2020_2984895 crossref_primary_10_1145_3530693 crossref_primary_10_1109_TSP_2023_3239799 crossref_primary_10_1109_TCNS_2022_3186653 crossref_primary_10_1007_s11227_019_02784_y crossref_primary_10_1109_TNNLS_2021_3125774 crossref_primary_10_1109_ACCESS_2021_3061995 crossref_primary_10_1109_TNET_2024_3350198 crossref_primary_10_1109_LCSYS_2023_3341100 crossref_primary_10_1016_j_engappai_2025_110626 crossref_primary_10_1137_16M1057000 crossref_primary_10_1109_TAC_2020_3033490 crossref_primary_10_1109_TASE_2020_2991042 crossref_primary_10_1007_s11431_024_2852_1 crossref_primary_10_1109_TSP_2025_3579628 crossref_primary_10_1007_s10107_019_01408_w crossref_primary_10_3390_en17051233 crossref_primary_10_1137_22M1490995 crossref_primary_10_1109_TSIPN_2020_2994008 crossref_primary_10_1109_TCNS_2025_3526555 crossref_primary_10_1007_s11227_020_03590_7 crossref_primary_10_1016_j_sigpro_2020_107529 crossref_primary_10_1109_TAC_2023_3261465 crossref_primary_10_1109_TCNS_2024_3354840 crossref_primary_10_1109_TPWRS_2022_3162329 crossref_primary_10_1007_s40998_021_00461_6 crossref_primary_10_1186_s13660_019_2145_0 crossref_primary_10_1287_opre_2022_0228 crossref_primary_10_1109_TCNS_2024_3462519 crossref_primary_10_1109_JIOT_2021_3049384 crossref_primary_10_1002_rnc_6048 crossref_primary_10_1109_TASE_2024_3435073 crossref_primary_10_1109_TCNS_2022_3213706 crossref_primary_10_1007_s10589_018_0034_y crossref_primary_10_1109_TSP_2023_3301621 |
| Cites_doi | 10.1016/0898-1221(76)90003-1 10.1109/GlobalSIP.2013.6736937 10.1515/9781400873173 10.1561/2200000016 10.1016/S1570-579X(01)80023-9 10.1109/TPDS.2017.2658620 10.1109/TAC.1986.1104412 10.1109/JSAC.2014.2328171 10.1137/140957639 10.1109/TWC.2010.06.090890 10.1111/j.1467-9868.2011.00783.x 10.1137/140998135 10.1109/TIT.2008.924723 10.1137/110836936 10.1007/BF01581204 10.1109/TSP.2016.2537261 10.1109/TSP.2016.2537271 10.1137/110849468 10.1137/15M1024950 10.1007/s10589-016-9828-y 10.1080/10556788.2012.700713 10.1109/JSAC.2002.1007390 10.1137/140990309 10.1007/s10957-013-0409-2 10.1137/070703983 10.1109/ICASSP.2015.7178689 10.1109/TSP.2015.2454476 10.1007/s10915-015-0048-x 10.1109/TSP.2016.2544743 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TCNS.2017.2657460 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2372-2533 |
| EndPage | 945 |
| ExternalDocumentID | 10_1109_TCNS_2017_2657460 7829332 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Science Foundation grantid: CCF-1526078 funderid: 10.13039/501100008982 – fundername: Air Force Office of Scientific Research grantid: 15RT0767 funderid: 10.13039/100000181 |
| GroupedDBID | 0R~ 4.4 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF M43 OCL PQQKQ RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c293t-30da3b82b9ed6cc78930bc49bec3649b13305b82331eabb5e24cbe34e2856e0d3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 46 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000445357100011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2325-5870 |
| IngestDate | Sun Jun 29 16:08:02 EDT 2025 Sat Nov 29 06:13:46 EST 2025 Tue Nov 18 20:45:15 EST 2025 Wed Aug 27 08:31:58 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c293t-30da3b82b9ed6cc78930bc49bec3649b13305b82331eabb5e24cbe34e2856e0d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-1263-9365 |
| PQID | 2117176493 |
| PQPubID | 2040410 |
| PageCount | 11 |
| ParticipantIDs | proquest_journals_2117176493 crossref_citationtrail_10_1109_TCNS_2017_2657460 crossref_primary_10_1109_TCNS_2017_2657460 ieee_primary_7829332 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-09-01 |
| PublicationDateYYYYMMDD | 2018-09-01 |
| PublicationDate_xml | – month: 09 year: 2018 text: 2018-09-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on control of network systems |
| PublicationTitleAbbrev | TCNS |
| PublicationYear | 2018 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | razaviyayn (ref50) 2014 ref14 hong (ref22) 2016 ref17 defazio (ref36) 2014 ref16 ref19 ref18 schmidt (ref40) 2013 agarwal (ref13) 2011 zhang (ref32) 2014 bertsekas (ref1) 1999 ref45 glowinski (ref15) 1975; 9 scutari (ref6) 2007 ref47 ref42 ref41 ref44 razaviyayn (ref38) 2013 ref49 hong (ref43) 0 ref7 zhang (ref26) 2010 (ref8) 2013 liu (ref12) 2014 ref4 ref3 ref5 ref35 ref34 richtárik (ref46) 2012 ref31 ref30 ref2 ref39 li (ref10) 2013 jiang (ref48) 2014 ho (ref9) 2013 ref24 ref23 sra (ref37) 2012 ref25 ref20 ref21 ref28 ref27 ref29 niu (ref11) 2011 wei (ref33) 2013 |
| References_xml | – ident: ref16 doi: 10.1016/0898-1221(76)90003-1 – year: 2013 ident: ref33 article-title: On the O(1/k) convergence of asynchronous distributed alternating direction method of multipliers doi: 10.1109/GlobalSIP.2013.6736937 – start-page: 345 year: 2007 ident: ref6 article-title: Asynchronous iterative water-filling for gaussian frequency-selective interference channels: A unified framework publication-title: Proc Inf Theory Appl Workshop – year: 2013 ident: ref38 article-title: A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks – ident: ref39 doi: 10.1515/9781400873173 – volume: 9 start-page: 41 year: 1975 ident: ref15 article-title: Sur l'approximation, par elements finis d'ordre un,et la resolution, par penalisation-dualite, d'une classe de problemes de dirichlet non lineares publication-title: Revue Franqaise d'Automatique Informatique et Rech Opirationelle – year: 1999 ident: ref1 publication-title: Parallel and Distributed Computation Numerical Methods – ident: ref35 doi: 10.1561/2200000016 – start-page: 1701 year: 2014 ident: ref32 article-title: Asynchronous distributed ADMM for consensus optimization publication-title: Proc 31st Int Conf Mach Learn – ident: ref4 doi: 10.1016/S1570-579X(01)80023-9 – year: 2013 ident: ref8 publication-title: Frontiers in Massive Data Analysis – year: 2016 ident: ref22 article-title: On the linear convergence of the alternating direction method of multipliers publication-title: Math Program Ser A – ident: ref24 doi: 10.1109/TPDS.2017.2658620 – ident: ref2 doi: 10.1109/TAC.1986.1104412 – ident: ref25 doi: 10.1109/JSAC.2014.2328171 – ident: ref41 doi: 10.1137/140957639 – year: 2013 ident: ref40 article-title: Minimizing finite sums with the stochastic average gradient publication-title: INRIA Paris France Tech Rep HAL 00860051 – ident: ref3 doi: 10.1109/TWC.2010.06.090890 – year: 2014 ident: ref48 article-title: Iteration bounds for finding the $\epsilon$ -stationary points for structured nonconvex optimization – start-page: 1646 year: 2014 ident: ref36 article-title: Saga: A fast incremental gradient method with support for non-strongly convex composite objectives publication-title: Proc Adv Neural Inf Process Syst – start-page: 693 year: 2011 ident: ref11 article-title: HOGWILD!: A lock-free approach to parallelizing stochastic gradient descent publication-title: Proc 24th Int Conf Neural Inf Process Syst – ident: ref49 doi: 10.1111/j.1467-9868.2011.00783.x – ident: ref18 doi: 10.1561/2200000016 – ident: ref31 doi: 10.1137/140998135 – start-page: 873 year: 2011 ident: ref13 article-title: Distributed delayed stochastic optimization publication-title: Proc Adv Neural Inf Process Syst – ident: ref5 doi: 10.1109/TIT.2008.924723 – ident: ref20 doi: 10.1137/110836936 – ident: ref17 doi: 10.1007/BF01581204 – year: 0 ident: ref43 article-title: Supplemental material for a distributed, asynchronous and incremental algorithm for nonconvex optimization: An ADMM based approach – year: 2013 ident: ref10 article-title: Distributed delayed proximal gradient methods publication-title: Proc NIPS Workshop Optim Mach Learn – year: 2012 ident: ref37 publication-title: Optimization for Machine Learning – start-page: 1440 year: 2014 ident: ref50 article-title: Parallel successive convex approximation for nonsmooth nonconvex optimization publication-title: Proc Neural Inf Process Syst – start-page: 469 year: 2014 ident: ref12 article-title: An asynchronous parallel stochastic coordinate descent algorithm publication-title: Proc Int Conf Mach Learn – ident: ref45 doi: 10.1109/TSP.2016.2537261 – year: 2012 ident: ref46 article-title: Alternating maximization: Unifying framework for 8 Sparse PCA formulations and efficient parallel codes – ident: ref44 doi: 10.1109/TSP.2016.2537271 – ident: ref21 doi: 10.1137/110849468 – ident: ref14 doi: 10.1137/15M1024950 – ident: ref27 doi: 10.1007/s10589-016-9828-y – ident: ref29 doi: 10.1080/10556788.2012.700713 – ident: ref7 doi: 10.1109/JSAC.2002.1007390 – ident: ref30 doi: 10.1137/140990309 – ident: ref42 doi: 10.1007/s10957-013-0409-2 – start-page: 10 year: 2010 ident: ref26 article-title: An alternating direction algorithm for nonnegative matrix factorization publication-title: Rice Univ Houston TX USA Tech Rep TR – ident: ref23 doi: 10.1137/070703983 – ident: ref47 doi: 10.1109/ICASSP.2015.7178689 – ident: ref28 doi: 10.1109/TSP.2015.2454476 – ident: ref19 doi: 10.1007/s10915-015-0048-x – ident: ref34 doi: 10.1109/TSP.2016.2544743 – start-page: 1223 year: 2013 ident: ref9 article-title: More effective distributed ML via a stale synchronous parallel parameter server publication-title: Proc Adv Neural Inf Process Syst |
| SSID | ssj0001255873 |
| Score | 2.3896616 |
| Snippet | The alternating direction method of multipliers (ADMM) has been popular for solving many signal processing problems, convex or nonconvex. In this paper, we... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 935 |
| SubjectTerms | Algorithm design and analysis Algorithms Alternating direction method of multipliers asynchronous optimization Control systems Delays distributed optimization Machine learning algorithms Nodes Optimization Signal processing Signal processing algorithms |
| Title | A Distributed, Asynchronous, and Incremental Algorithm for Nonconvex Optimization: An ADMM Approach |
| URI | https://ieeexplore.ieee.org/document/7829332 https://www.proquest.com/docview/2117176493 |
| Volume | 5 |
| WOSCitedRecordID | wos000445357100011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2372-2533 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001255873 issn: 2325-5870 databaseCode: RIE dateStart: 20140101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NT8IwFG-UeNCDX2hE0fTgyTAY67Zu3haQeBA0EQ23ZWsfSgLDwDD63_vaFSTRmHjZlqxtuv22996vfR-EXIqUS2RcroXGx9ByIUislOFBwtBxk2AYSr17_nzHe71gMAgfNkhtFQsDANr5DOrqUu_ly6lYqKWyBmoz5N8ocDc594tYrbX1FM8LODMbl007bPRbvUflu8Xrju9xVyeh_FY9upbKDwGstUpn73_z2Se7xnqkUQH3AdmA7JDsrOUULBMR0bZKhqvqWIGs0Wj-mQmVARcpfo0mmaQoEopFQTXS-GU6G-WvE4rGK-1NM-2F_kHvUZJMTIjmNY0yGrW7XRqZBORH5Klz02_dWqaSgiVwhrnFbJmwNHDSEKQvBEcjxU6FGyKAzMcTElXbw_uMNSFJUw8cV6TAXHACzwdbsmNSyqYZnBCKKh45SNNJhqj8E4-HHk84hMJXik7askLs5UuOhUkzrqpdjGNNN-wwVrjECpfY4FIhV6sub0WOjb8alxUQq4YGgwqpLpGMzV84j5HcIlvFB2Snv_c6I9s4dlD4jFVJKZ8t4Jxsifd8NJ9d6A_sCwsezwk |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1RT9swED6hgjR4gI2CKCvDD3tCDU3jJI73FhVQ0doMad3EW5TY1w0JUtQWBP-es2MKEtMkXpJIsRMnX3J33_l8B_BVlUIT4wo9Mj4mXohJ4ZWcNhonQVgkE6nt7Pnvociy5PJSXqxAZ7kWBhFt8Bkem0M7l6-n6s64yrqkzYh_k8BdNZWzZL1a65VHJYoSwd3UZc-X3XE_-2mit8RxEEcitGkoX5SPrabyRgRbvXK29b4RfYRNZz-ytAb8E6xgtQ0br7IKNkGl7MSkwzWVrFB3WDp_rJTJgUskv8OKSjMSCrVb0Fzp-s90drX4e8PIfGXZtLJx6A_sB8mSG7dI8xtLK5aejEYsdSnId-DX2em4P_BcLQVP0QgXHvd1wcskKCXqWClBZopfqlAShDymHVFVP6LznPewKMsIg1CVyEMMkihGX_NdaFTTCveAkZInFtILigmp_yISMhKFQKlio-q0r1vgP7_kXLlE46bexXVuCYcvc4NLbnDJHS4tOFp2ua2zbPyvcdMAsWzoMGhB-xnJ3P2H85zoLfFVekC-_-9eh_BhMB4N8-F59v0zrNN9kjqCrA2NxewOD2BN3S-u5rMv9mN7Agv10lg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Distributed%2C+Asynchronous%2C+and+Incremental+Algorithm+for+Nonconvex+Optimization%3A+An+ADMM+Approach&rft.jtitle=IEEE+transactions+on+control+of+network+systems&rft.au=Hong%2C+Mingyi&rft.date=2018-09-01&rft.pub=IEEE&rft.eissn=2372-2533&rft.volume=5&rft.issue=3&rft.spage=935&rft.epage=945&rft_id=info:doi/10.1109%2FTCNS.2017.2657460&rft.externalDocID=7829332 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2325-5870&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2325-5870&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2325-5870&client=summon |