A Fast Distributed Asynchronous Newton-Based Optimization Algorithm
One of the most important problems in the field of distributed optimization is the problem of minimizing a sum of local convex objective functions over a networked system. Most of the existing work in this area focuses on developing distributed algorithms in a synchronous setting under the presence...
Uloženo v:
| Vydáno v: | IEEE transactions on automatic control Ročník 65; číslo 7; s. 2769 - 2784 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
IEEE
01.07.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 0018-9286, 1558-2523 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | One of the most important problems in the field of distributed optimization is the problem of minimizing a sum of local convex objective functions over a networked system. Most of the existing work in this area focuses on developing distributed algorithms in a synchronous setting under the presence of a central clock, where the agents need to wait for the slowest one to finish the update, before proceeding to the next iterate. Asynchronous distributed algorithms remove the need for a central coordinator, reduce the synchronization wait, and allow some agents to compute faster and execute more iterations. In the asynchronous setting, the only known algorithms for solving this problem could achieve an either linear or sublinear rate of convergence. In this paper, we build upon the existing literature to develop and analyze an asynchronous Newton-based method to solve a penalized version of the problem. We show that this algorithm guarantees almost sure convergence with a global linear and local quadratic rate in expectation. Numerical studies confirm the superior performance of our algorithm against other asynchronous methods. |
|---|---|
| AbstractList | One of the most important problems in the field of distributed optimization is the problem of minimizing a sum of local convex objective functions over a networked system. Most of the existing work in this area focuses on developing distributed algorithms in a synchronous setting under the presence of a central clock, where the agents need to wait for the slowest one to finish the update, before proceeding to the next iterate. Asynchronous distributed algorithms remove the need for a central coordinator, reduce the synchronization wait, and allow some agents to compute faster and execute more iterations. In the asynchronous setting, the only known algorithms for solving this problem could achieve an either linear or sublinear rate of convergence. In this paper, we build upon the existing literature to develop and analyze an asynchronous Newton-based method to solve a penalized version of the problem. We show that this algorithm guarantees almost sure convergence with a global linear and local quadratic rate in expectation. Numerical studies confirm the superior performance of our algorithm against other asynchronous methods. |
| Author | Mansoori, Fatemeh Wei, Ermin |
| Author_xml | – sequence: 1 givenname: Fatemeh orcidid: 0000-0002-6574-4727 surname: Mansoori fullname: Mansoori, Fatemeh email: fatemehmansoori2019@u.northwestern.edu organization: Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA – sequence: 2 givenname: Ermin orcidid: 0000-0002-8035-484X surname: Wei fullname: Wei, Ermin email: ermin.wei@northwestern.edu organization: Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA |
| BookMark | eNp9kD1PwzAQhi1UJNrCjsQSiTnBduzEGUOggFTRpcyW4w_qqo2L7QiVX09KKwYGptPp3udO90zAqHOdBuAawQwhWN0t6ybDEFUZrvK8gOUZGCNKWYopzkdgDCFiaYVZcQEmIayHtiAEjUFTJzMRYvJgQ_S27aNWSR32nVx517k-JK_6M7ouvRdhmCx20W7tl4jWdUm9eXfextX2EpwbsQn66lSn4G32uGye0_ni6aWp56nEFYopZcqYtiSKEqgwpZBSjQ0kQhhayBYSIpQRRiupJEFKFogJJaURqFUCY5pPwe1x7867j16HyNeu991wkmOCGEEVK-CQgseU9C4Erw3febsVfs8R5AdVfFDFD6r4SdWAFH8QaePPk9ELu_kPvDmCVmv9e4eVrCJlnn8D2TN5FQ |
| CODEN | IETAA9 |
| CitedBy_id | crossref_primary_10_1080_00207179_2021_1970816 crossref_primary_10_1016_j_automatica_2021_110156 crossref_primary_10_1109_TCNS_2021_3070663 crossref_primary_10_1016_j_automatica_2021_109886 crossref_primary_10_1016_j_automatica_2024_111629 crossref_primary_10_1016_j_automatica_2020_109286 crossref_primary_10_1109_TCYB_2020_3011819 crossref_primary_10_1109_TSIPN_2021_3083975 crossref_primary_10_3390_fractalfract8120721 crossref_primary_10_1007_s11431_024_2852_1 crossref_primary_10_1109_TAC_2021_3123244 crossref_primary_10_1109_TAC_2024_3439652 crossref_primary_10_1109_TCNS_2023_3285865 crossref_primary_10_1109_TETCI_2024_3437249 crossref_primary_10_1109_TNNLS_2022_3192346 crossref_primary_10_1109_TSMC_2024_3496488 crossref_primary_10_1109_TSMC_2022_3228780 crossref_primary_10_1109_TAC_2024_3360287 crossref_primary_10_1109_TSP_2023_3240652 crossref_primary_10_1109_MRA_2024_3352852 crossref_primary_10_1109_MRA_2024_3358718 crossref_primary_10_1109_TSIPN_2022_3203860 crossref_primary_10_1016_j_neunet_2023_06_001 crossref_primary_10_1109_TAC_2023_3261465 crossref_primary_10_1109_TASLP_2022_3224256 crossref_primary_10_1109_TSP_2024_3424436 crossref_primary_10_1109_TAC_2023_3244904 crossref_primary_10_1016_j_cam_2024_116250 crossref_primary_10_1016_j_asoc_2024_111976 crossref_primary_10_1109_TPWRS_2021_3056543 crossref_primary_10_1109_TASE_2025_3550740 crossref_primary_10_1109_TAC_2022_3173171 crossref_primary_10_1109_LCSYS_2021_3086388 |
| Cites_doi | 10.1109/TAC.2013.2253218 10.1109/TSP.2016.2537261 10.1109/MLSP.2014.6958862 10.1145/1961189.1961199 10.1109/CDC.2009.5400289 10.1109/TSP.2016.2537271 10.1109/JSTSP.2011.2120593 10.1007/s10107-015-0901-6 10.1109/JSTSP.2011.2118740 10.1109/TAC.1986.1104412 10.1109/TAC.2011.2161027 10.1137/S0036144504446096 10.1137/15M1024950 10.1109/TSP.2015.2449255 10.1561/2200000016 10.1007/978-3-642-12598-0_5 10.1109/CDC.2017.8264076 10.1109/TAC.2008.2009515 10.1109/GlobalSIP.2013.6736937 10.1109/TSIPN.2016.2524588 10.1109/TAC.2014.2298712 10.1137/16M1080173 10.1109/TAC.2003.812781 10.1109/JPROC.2010.2052531 10.1109/ACSSC.2016.7869154 10.1017/CBO9780511804441 10.1109/TAC.2015.2512043 10.1109/CDC.2012.6426375 10.1137/130943170 10.1109/TAC.1983.1103183 10.1017/CBO9780511810817 10.1145/984622.984626 10.1109/TSIPN.2017.2695121 10.1137/14096668X 10.1137/1.9780898718003 10.1109/GlobalSIP.2016.7905906 10.1109/TIT.2006.874516 10.1109/CDC.2012.6425904 10.1109/TSP.2013.2254478 10.1109/TAC.2010.2079650 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D |
| DOI | 10.1109/TAC.2019.2933607 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-2523 |
| EndPage | 2784 |
| ExternalDocumentID | 10_1109_TAC_2019_2933607 8789473 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Defense Advanced Research Projects Agency under Contract Langrange grantid: HR-001117S0039 |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 VJK ~02 AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c291t-58dffb74d540d255055e2f04aaf56cb044adfafedcdc41dc618adccfa1bda2253 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 36 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000543961200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0018-9286 |
| IngestDate | Mon Jun 30 10:17:55 EDT 2025 Tue Nov 18 22:43:12 EST 2025 Sat Nov 29 05:40:56 EST 2025 Wed Aug 27 02:38:19 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 7 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-58dffb74d540d255055e2f04aaf56cb044adfafedcdc41dc618adccfa1bda2253 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-8035-484X 0000-0002-6574-4727 |
| PQID | 2418419860 |
| PQPubID | 85475 |
| PageCount | 16 |
| ParticipantIDs | proquest_journals_2418419860 crossref_primary_10_1109_TAC_2019_2933607 ieee_primary_8789473 crossref_citationtrail_10_1109_TAC_2019_2933607 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-07-01 |
| PublicationDateYYYYMMDD | 2020-07-01 |
| PublicationDate_xml | – month: 07 year: 2020 text: 2020-07-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on automatic control |
| PublicationTitleAbbrev | TAC |
| PublicationYear | 2020 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 jerinkic (ref2) 2017 ref15 ref52 ref11 ref54 ref10 ref17 ref16 ref19 ref18 ref51 ref50 zanella (ref55) 0 ref46 ref45 bertsekas (ref4) 1989; 23 ref47 ref41 ref43 ref49 ref8 cottle (ref14) 1992; 60 ref7 ref9 ref3 xiao (ref53) 0 ref5 ref40 hajinezhad (ref21) 0 ref36 ref31 ref33 ref39 ref38 nedi? (ref32) 2011; 56 tsitsiklis (ref48) 1984 mokhtari (ref30) 2015 ref24 ref23 ref26 ref25 ref20 bof (ref6) 2017 assran (ref1) 2018 ref22 nocedal (ref35) 2006 shamir (ref42) 0; 32 ref28 ref27 polyak (ref37) 1987 mokhtari (ref29) 2015 nesterov (ref34) 2013; 87 nedi? (ref44) 2011; 5 |
| References_xml | – ident: ref51 doi: 10.1109/TAC.2013.2253218 – ident: ref13 doi: 10.1109/TSP.2016.2537261 – ident: ref19 doi: 10.1109/MLSP.2014.6958862 – year: 2015 ident: ref29 article-title: Network Newton-Part I: Algorithm and convergence publication-title: arXiv preprint arXiv 1504 06017 – start-page: 63 year: 0 ident: ref53 article-title: A scheme for robust distributed sensor fusion based on average consensus publication-title: Proc 4th Int Symp Inf Process Sens Netw – ident: ref11 doi: 10.1145/1961189.1961199 – ident: ref24 doi: 10.1109/CDC.2009.5400289 – ident: ref12 doi: 10.1109/TSP.2016.2537271 – year: 2006 ident: ref35 publication-title: Nonlinear Equations – start-page: 3215 year: 0 ident: ref21 article-title: NESTT: A nonconvex primal-dual splitting method for distributed and stochastic optimization publication-title: Proc Int Conf Neural Inf Process – ident: ref28 doi: 10.1109/JSTSP.2011.2120593 – ident: ref40 doi: 10.1007/s10107-015-0901-6 – volume: 5 start-page: 772 year: 2011 ident: ref44 article-title: Distributed asynchronous constrained stochastic optimization publication-title: IEEE J Sel Topics Signal Process doi: 10.1109/JSTSP.2011.2118740 – ident: ref47 doi: 10.1109/TAC.1986.1104412 – ident: ref17 doi: 10.1109/TAC.2011.2161027 – ident: ref20 doi: 10.1137/S0036144504446096 – ident: ref36 doi: 10.1137/15M1024950 – ident: ref26 doi: 10.1109/TSP.2015.2449255 – volume: 60 year: 1992 ident: ref14 publication-title: The Linear Complementarity Problem – ident: ref9 doi: 10.1561/2200000016 – ident: ref39 doi: 10.1007/978-3-642-12598-0_5 – year: 1987 ident: ref37 publication-title: Introduction to Optimization – ident: ref27 doi: 10.1109/CDC.2017.8264076 – ident: ref33 doi: 10.1109/TAC.2008.2009515 – ident: ref50 doi: 10.1109/GlobalSIP.2013.6736937 – ident: ref15 doi: 10.1109/TSIPN.2016.2524588 – ident: ref25 doi: 10.1109/TAC.2014.2298712 – ident: ref7 doi: 10.1137/16M1080173 – year: 2017 ident: ref6 article-title: Newton-Raphson consensus under asynchronous and lossy communications for peer-to-peer networks publication-title: arXiv preprint arXiv 1707 09562 – year: 1984 ident: ref48 article-title: Problems in decentralized decision making and computation – ident: ref23 doi: 10.1109/TAC.2003.812781 – year: 2017 ident: ref2 article-title: Distributed second order methods with variable number of working nodes publication-title: arXiv 1709 01307 – ident: ref16 doi: 10.1109/JPROC.2010.2052531 – ident: ref45 doi: 10.1109/ACSSC.2016.7869154 – ident: ref10 doi: 10.1017/CBO9780511804441 – ident: ref5 doi: 10.1109/TAC.2015.2512043 – ident: ref46 doi: 10.1109/CDC.2012.6426375 – ident: ref54 doi: 10.1137/130943170 – year: 2015 ident: ref30 article-title: Network Newton-Part II: Convergence rate and implementation publication-title: Arxiv preprint Arxiv 1504 06010 – ident: ref3 doi: 10.1109/TAC.1983.1103183 – volume: 87 year: 2013 ident: ref34 publication-title: Introductory Lectures on Convex Optimization A Basic Course – ident: ref22 doi: 10.1017/CBO9780511810817 – ident: ref38 doi: 10.1145/984622.984626 – ident: ref52 doi: 10.1109/TSIPN.2017.2695121 – ident: ref43 doi: 10.1137/14096668X – start-page: 5917 year: 0 ident: ref55 article-title: Newton-Raphson consensus for distributed convex optimization publication-title: Proc 50th IEEE Conf Decis Control/Eur Control Conf – volume: 32 start-page: 1000 year: 0 ident: ref42 article-title: Communication-efficient distributed optimization using an approximate Newton-type method publication-title: Proc 31st Int Conf Mach Learn – ident: ref41 doi: 10.1137/1.9780898718003 – year: 2018 ident: ref1 article-title: Asynchronous subgradient-push publication-title: arXiv preprint arXiv 1803 08950 – ident: ref18 doi: 10.1109/GlobalSIP.2016.7905906 – ident: ref8 doi: 10.1109/TIT.2006.874516 – ident: ref49 doi: 10.1109/CDC.2012.6425904 – ident: ref31 doi: 10.1109/TSP.2013.2254478 – volume: 23 year: 1989 ident: ref4 publication-title: Parallel and Distributed Computation Numerical Methods – volume: 56 start-page: 1337 year: 2011 ident: ref32 article-title: Asynchronous broadcast-based optimization over a network publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.2010.2079650 |
| SSID | ssj0016441 |
| Score | 2.543574 |
| Snippet | One of the most important problems in the field of distributed optimization is the problem of minimizing a sum of local convex objective functions over a... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 2769 |
| SubjectTerms | Agents and autonomous systems Algorithms Approximation algorithms asynchronous algorithms Clocks Convergence Delays Linear programming network analysis and control Optimization optimization algorithms Symmetric matrices Synchronism |
| Title | A Fast Distributed Asynchronous Newton-Based Optimization Algorithm |
| URI | https://ieeexplore.ieee.org/document/8789473 https://www.proquest.com/docview/2418419860 |
| Volume | 65 |
| WOSCitedRecordID | wos000543961200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 0018-9286 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB7a4kEPvqpYrZKDF8G0eWyS3WOsFg9SPVToLWx2drXQhzSp4L93N0lDQRG8JbALy2zmlZn5PoBrisxFStF2HK5sEiKzU59Qm-tTew4PeVrizD5FoxGdTNhLA27rWRgpZdF8Jnvmsajl41Ksza-yPo0oI5HfhGYUheWsVl0xMH69tLpagT1alyQd1h_HA9PDxXratfmhIY7dckEFp8oPQ1x4l-HB_851CPtVFGnF5bUfQUMujmFvC1uwDYPYGvIst-4NMq4htZJoxdnXQhg0XJ3uW9q8GfrgO-3G0HrWlmNejWRa8extuZrm7_MTeB0-jAePdsWYYAuPubkdUFQqjQjqOAw9k3wE0lMO4VwFoUgdQjgqriQKFMRFEbqUoxCKuylyrdn-KbQWy4U8A0ua1xB56gecRGby3GOKc6ojhFQQknagvxFiIio4ccNqMUuKtMJhiRZ7YsSeVGLvwE2946OE0vhjbduIuV5XSbgD3c09JZWuZYmOQShxGQ2d8993XcCuZ7Lkosm2C618tZaXsCM-82m2uio-o29YYcYq |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB58gXrwLa7PHrwI1u0jbZNjXV0U19XDCt5KmklU0F3ZrYL_3kxbi6AI3lpIIEw6r87M9wEcchQ-co6u50njshiFm4eMu9KeOvBkLPMKZ7aX9Pv8_l7cTsFxMwujtS6bz_QJPZa1fBypN_pV1uYJFywJp2GWmLOialqrqRmQZ6_srlXhgDdFSU-0B2mHurjEiXVuYUzUsd-cUMmq8sMUl_6lu_y_k63AUh1HOml18aswpYdrsPgNXXAdOqnTlZPCOSNsXKK10uikk4-hIjxcm_A71sARgfCpdWTo3Fjb8VIPZTrp88No_FQ8vmzAXfd80Llwa84EVwXCL9yIozF5wtBGYhhQ-hHpwHhMShPFKvcYk2ik0ahQMR9V7HOJShnp5yitboebMDMcDfUWOJpeY5R5GEmW0Ox5IIyU3MYIuWIsb0H7S4iZqgHFidfiOSsTC09kVuwZiT2rxd6Co2bHawWm8cfadRJzs66WcAt2v-4pq7VtktkohDNf8Njb_n3XAcxfDK57We-yf7UDCwHlzGXL7S7MFOM3vQdz6r14moz3y0_qE9tByXU |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Fast+Distributed+Asynchronous+Newton-Based+Optimization+Algorithm&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Mansoori%2C+Fatemeh&rft.au=Wei%2C+Ermin&rft.date=2020-07-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9286&rft.eissn=1558-2523&rft.volume=65&rft.issue=7&rft.spage=2769&rft_id=info:doi/10.1109%2FTAC.2019.2933607&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon |