Asynchronous Optimization Over Graphs: Linear Convergence Under Error Bound Conditions

We consider convex and nonconvex constrained optimization with a partially separable objective function: Agents minimize the sum of local objective functions, each of which is known only by the associated agent and depends on the variables of that agent and those of a few others. This partitioned se...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on automatic control Ročník 66; číslo 10; s. 4604 - 4619
Hlavní autori:	Cannelli, Loris, Facchinei, Francisco, Scutari, Gesualdo, Kungurtsev, Vyacheslav
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.10.2021 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Algorithms Asynchronous algorithms Convergence Convexity Delays error bounds Indexes Linear programming linear rate multiagent systems Nickel nonconvex optimization Optimization Partitioning algorithms
ISSN:	0018-9286, 1558-2523
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	We consider convex and nonconvex constrained optimization with a partially separable objective function: Agents minimize the sum of local objective functions, each of which is known only by the associated agent and depends on the variables of that agent and those of a few others. This partitioned setting arises in several applications of practical interest. We propose what is, to the best of our knowledge, the first distributed, asynchronous algorithm with rate guarantees for this class of problems. When the objective function is nonconvex, the algorithm provably converges to a stationary solution at a sublinear rate whereas linear rate is achieved under the renowned Luo-Tseng error bound condition (which is less stringent than strong convexity). Numerical results on matrix completion and LASSO problems show the effectiveness of our method.
AbstractList	We consider convex and nonconvex constrained optimization with a partially separable objective function: Agents minimize the sum of local objective functions, each of which is known only by the associated agent and depends on the variables of that agent and those of a few others. This partitioned setting arises in several applications of practical interest. We propose what is, to the best of our knowledge, the first distributed, asynchronous algorithm with rate guarantees for this class of problems. When the objective function is nonconvex, the algorithm provably converges to a stationary solution at a sublinear rate whereas linear rate is achieved under the renowned Luo-Tseng error bound condition (which is less stringent than strong convexity). Numerical results on matrix completion and LASSO problems show the effectiveness of our method.
Author	Scutari, Gesualdo Facchinei, Francisco Cannelli, Loris Kungurtsev, Vyacheslav
Author_xml	– sequence: 1 givenname: Loris orcidid: 0000-0002-7161-6414 surname: Cannelli fullname: Cannelli, Loris email: loris.cannelli@idsia.ch organization: Istituto Dalle Molle di studi sull’Intelligenza Artificiale (IDSIA), USI/SUPSI, Lugano, Switzerland – sequence: 2 givenname: Francisco orcidid: 0000-0002-7714-1210 surname: Facchinei fullname: Facchinei, Francisco email: facchinei@dis.uniroma1.it organization: Department of Computer, Control, and Management Engineering, University of Rome La Sapienza, Rome, Italy – sequence: 3 givenname: Gesualdo orcidid: 0000-0002-6453-6870 surname: Scutari fullname: Scutari, Gesualdo email: gscutari@purdue.edu organization: School of Industrial Engineering, Purdue University, West-Lafayette, IN, USA – sequence: 4 givenname: Vyacheslav orcidid: 0000-0003-2229-8824 surname: Kungurtsev fullname: Kungurtsev, Vyacheslav email: vyacheslav.kungurtsev@fel.cvut.cz organization: Department of Computer Science, Czech Technical University, Prague, Czech Republic
BookMark	eNp9kM9LwzAUx4NMcJveBS8Fz5350aSptznmFAa7bF5D2iYuY0tq0grzrzd1w4MHT4_33vfzHnxGYGCdVQDcIjhBCBYP6-lsgiGGEwIJyQp4AYaIUp5iiskADCFEPC0wZ1dgFMIutizL0BC8TcPRVlvvrOtCsmpaczBfsjXOJqtP5ZOFl802PCZLY5X0yczZOH1XtlLJxtYxMPfe-eTJdbbut7Xp2XANLrXcB3VzrmOweZ6vZy_pcrV4nU2XaUUIaVOOGSRUMoVyWuSFxqjGpaZaZxXCkuKK5jnTdclKXZYac8h4iQoCkdQ5YbQmY3B_utt499Gp0Iqd67yNLwWm8SbHlOUxBU-pyrsQvNKi8eYg_VEgKHp7ItoTvT1xthcR9gepTPvjpfXS7P8D706gUUr9_ikw4RnJyDc7iH4L
CODEN	IETAA9
CitedBy_id	crossref_primary_10_1016_j_ifacol_2022_07_249 crossref_primary_10_1080_10556788_2021_2023524 crossref_primary_10_1109_TAC_2024_3522188 crossref_primary_10_1109_TCNS_2022_3203366 crossref_primary_10_1109_LCSYS_2023_3270347 crossref_primary_10_1088_1361_6420_acbdb9 crossref_primary_10_1109_LCSYS_2021_3082800 crossref_primary_10_1137_20M1353149 crossref_primary_10_1109_TAC_2020_3033490 crossref_primary_10_1109_TAC_2023_3261465
Cites_doi	10.1109/TAC.2017.2730481 10.1109/GlobalSIP.2018.8646514 10.1109/CDC.2013.6760448 10.1109/CDC.2016.7798262 10.1137/14096668X 10.1109/TAC.2010.2079650 10.1109/TSP.2015.2399858 10.1111/j.2517-6161.1996.tb02080.x 10.1109/TAC.2016.2607023 10.1137/0801036 10.1109/TSP.2016.2537271 10.1137/15M1024950 10.1109/TAC.2015.2512043 10.1109/Allerton.2011.6120272 10.1109/TAC.2020.3033490 10.1109/TSMC.2014.2332306 10.1109/TAC.2020.2977940 10.1109/LSP.2012.2207719 10.1137/140961134 10.1109/TSIPN.2017.2695121 10.1016/j.neucom.2015.12.017 10.1007/s40305-013-0015-x 10.1109/JSTSP.2011.2118740 10.1137/0330025 10.1109/TSP.2014.2385046 10.1007/BF02096261 10.1109/CDC.2013.6760336 10.1109/GlobalSIP.2013.6736937 10.1287/moor.2017.0889 10.1016/j.automatica.2015.11.014 10.1109/TAC.1986.1104412 10.1007/s10107-007-0170-0 10.1109/TSIPN.2016.2593896 10.1007/s10107-019-01408-w 10.1109/TPWRS.2012.2219629 10.1109/ACC.2012.6315289 10.1109/TCNS.2017.2657460 10.1109/HPCC/SmartCity/DSS.2019.00016
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
DOI	10.1109/TAC.2020.3033490
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-2523
EndPage	4619
ExternalDocumentID	10_1109_TAC_2020_3033490 9238434
Genre	orig-research
GrantInformation_xml	– fundername: ARO grantid: W911NF1810238 – fundername: OP VVV grantid: CZ.02.1.01/0.0/0.0/16_019/0000765 – fundername: National Science Foundation; USA NSF grantid: CIF 1719205; CMMI 1832688 funderid: 10.13039/100000001
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 VJK ~02 AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c333t-826035a6e175979f21d2bf5ff4c12a52c5776fdb6bfbbf28068b19301af7365d3
IEDL.DBID	RIE
ISICitedReferencesCount	13
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000698859900013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9286
IngestDate	Mon Jun 30 10:17:49 EDT 2025 Tue Nov 18 20:48:39 EST 2025 Sat Nov 29 05:41:00 EST 2025 Wed Aug 27 02:26:51 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	10
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c333t-826035a6e175979f21d2bf5ff4c12a52c5776fdb6bfbbf28068b19301af7365d3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-7161-6414 0000-0002-7714-1210 0000-0003-2229-8824 0000-0002-6453-6870
OpenAccessLink	https://hdl.handle.net/11573/1474315
PQID	2575982567
PQPubID	85475
PageCount	16
ParticipantIDs	proquest_journals_2575982567 ieee_primary_9238434 crossref_citationtrail_10_1109_TAC_2020_3033490 crossref_primary_10_1109_TAC_2020_3033490
PublicationCentury	2000
PublicationDate	2021-10-01
PublicationDateYYYYMMDD	2021-10-01
PublicationDate_xml	– month: 10 year: 2021 text: 2021-10-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on automatic control
PublicationTitleAbbrev	TAC
PublicationYear	2021
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 niu (ref18) 2011 ref10 bof (ref38) 2017 ref50 ref45 doan (ref33) 2017 ref48 ref47 ref42 nedi? (ref23) 2011; 5 ref43 davis (ref16) 2016 ref8 ref7 ref9 ref4 ref3 ref6 ref5 sun (ref11) 2020 nedi? (ref27) 2011; 56 ref35 ref34 ref37 ref36 ref31 ref30 zhu (ref41) 2018 ref32 ref2 ref1 ref39 zhang (ref44) 2014 lian (ref19) 2015 rennie (ref46) 2005 ref24 shah (ref40) 2020 ref26 ref25 ref20 ref22 ref21 ref28 ref29 bertsekas (ref17) 1989; 23 cannelli (ref49) 2017
References_xml	– ident: ref25 doi: 10.1109/TAC.2017.2730481 – ident: ref43 doi: 10.1109/GlobalSIP.2018.8646514 – ident: ref20 doi: 10.1109/CDC.2013.6760448 – ident: ref26 doi: 10.1109/CDC.2016.7798262 – ident: ref12 doi: 10.1137/14096668X – year: 2020 ident: ref11 article-title: Distributed optimization based on gradient-tracking revisited: Enhancing convergence rate via surrogation – year: 2017 ident: ref49 article-title: Asynchronous parallel algorithms for nonconvex big-data optimization Part II: Complexity and numerical results – volume: 56 start-page: 1337 year: 2011 ident: ref27 article-title: Asynchronous broadcast-based convex optimization over a network publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.2010.2079650 – ident: ref48 doi: 10.1109/TSP.2015.2399858 – ident: ref47 doi: 10.1111/j.2517-6161.1996.tb02080.x – start-page: 2719 year: 2015 ident: ref19 article-title: Asynchronous parallel stochastic gradient for nonconvex optimization publication-title: Proc Adv Neural Inf Process Syst 28 – ident: ref24 doi: 10.1109/TAC.2016.2607023 – year: 2020 ident: ref40 article-title: Linearly convergent asynchronous distributed ADMM via Markov sampling – volume: 23 year: 1989 ident: ref17 publication-title: Parallel and Distributed Computation Numerical Methods – ident: ref6 doi: 10.1137/0801036 – ident: ref42 doi: 10.1109/TSP.2016.2537271 – ident: ref37 doi: 10.1137/15M1024950 – ident: ref22 doi: 10.1109/TAC.2015.2512043 – ident: ref31 doi: 10.1109/Allerton.2011.6120272 – ident: ref50 doi: 10.1109/TAC.2020.3033490 – ident: ref28 doi: 10.1109/TSMC.2014.2332306 – ident: ref10 doi: 10.1109/TAC.2020.2977940 – ident: ref3 doi: 10.1109/LSP.2012.2207719 – ident: ref14 doi: 10.1137/140961134 – ident: ref36 doi: 10.1109/TSIPN.2017.2695121 – ident: ref29 doi: 10.1016/j.neucom.2015.12.017 – ident: ref8 doi: 10.1007/s40305-013-0015-x – year: 2017 ident: ref38 article-title: Newton-Raphson consensus under asynchronous and lossy communications for peer-to-peer networks – volume: 5 start-page: 772 year: 2011 ident: ref23 article-title: Distributed asynchronous constrained stochastic optimization publication-title: IEEE J Sel Top Signal Process doi: 10.1109/JSTSP.2011.2118740 – ident: ref5 doi: 10.1137/0330025 – ident: ref34 doi: 10.1109/TSP.2014.2385046 – ident: ref4 doi: 10.1007/BF02096261 – year: 2018 ident: ref41 article-title: A block-wise, asynchronous and distributed ADMM algorithm for general form consensus optimization – year: 2017 ident: ref33 article-title: Impact of communication delays on the convergence rate of distributed optimization algorithms – ident: ref1 doi: 10.1109/CDC.2013.6760336 – start-page: 693 year: 2011 ident: ref18 article-title: HOGWILD: A lock-free approach to parallelizing stochastic gradient descent publication-title: Proc Adv Neural Inf Process Syst 24 – start-page: 1329 year: 2005 ident: ref46 article-title: Maximum-margin matrix factorization publication-title: Proc Adv Neural Inf Process Syst 17 – ident: ref21 doi: 10.1109/GlobalSIP.2013.6736937 – ident: ref9 doi: 10.1287/moor.2017.0889 – ident: ref32 doi: 10.1016/j.automatica.2015.11.014 – ident: ref13 doi: 10.1109/TAC.1986.1104412 – ident: ref7 doi: 10.1007/s10107-007-0170-0 – ident: ref35 doi: 10.1109/TSIPN.2016.2593896 – ident: ref15 doi: 10.1007/s10107-019-01408-w – ident: ref2 doi: 10.1109/TPWRS.2012.2219629 – ident: ref30 doi: 10.1109/ACC.2012.6315289 – ident: ref39 doi: 10.1109/TCNS.2017.2657460 – start-page: 226 year: 2016 ident: ref16 article-title: The sound of APALM clapping: Faster nonsmooth nonconvex optimization with stochastic asynchronous PALM publication-title: Proc 30th Conf Adv Neural Inf Process Syst 29 – start-page: 1701 year: 2014 ident: ref44 article-title: Asynchronous distributed ADMM for consensus optimization publication-title: Proc 31st Int Conf Mach Learn – ident: ref45 doi: 10.1109/HPCC/SmartCity/DSS.2019.00016
SSID	ssj0016441
Score	2.470869
Snippet	We consider convex and nonconvex constrained optimization with a partially separable objective function: Agents minimize the sum of local objective functions,...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	4604
SubjectTerms	Algorithms Asynchronous algorithms Convergence Convexity Delays error bounds Indexes Linear programming linear rate multiagent systems Nickel nonconvex optimization Optimization Partitioning algorithms
Title	Asynchronous Optimization Over Graphs: Linear Convergence Under Error Bound Conditions
URI	https://ieeexplore.ieee.org/document/9238434 https://www.proquest.com/docview/2575982567
Volume	66
WOSCitedRecordID	wos000698859900013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 0018-9286 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5q8SL4rGK1yh68CMYm2bzWWy2tHqT1UEtvYbMPELSVpBX8985u0lBQBG-B7EKY2Z2ZL_P4AK58qhNlqsRdzaUTxFQ5WchdR1ERcS5EwCxp3_QpHo2S2Yw9N-Cm7oVRStniM3VrHm0uXy7Eyvwq62IwkgQ02IKtOI7KXq06Y2D8eml18QL7SZ2SdFl30usjEPQRn7qUBsb6brggy6nywxBb7zLc_993HcBeFUWSXqn2Q2io-RHsbMwWbMG0V3zNhRl9i9iejNEyvFctl2SMx5c8mEnVxR1BMIqHnfRN9bltxFTEciGRQZ4vcnJvaJfMW1nWdh3Dy3Aw6T86FYmCIyilSwfhg0tDHimME1jMtO9JP9Oh1oHwfB76IkRZaplFmc4ybfKsSYZBnetxHdMolPQEmvPFXJ0CYdLTnkwSHcQ6CKnOQpYIDCAEZ5wx6bahu5ZrKqoJ44bo4i21SMNlKWoiNZpIK0204bre8VFO1_hjbctIvl5XCb0NnbXq0ur6FalvaEcR-0bx2e-7zmHXmIeyDIehL-pAc5mv1AVsi8_la5Ff2tP1DWXczA0
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1ZS8NAEB7qAeqDVxXruQ--CMYmu7nWt1q8sLY-VOlb2OwBgraStoL_3p1tDIIi-BbILoSd2Zn5MscHcEyZSTVWiftGKC9MmPbySPieZjIWQsqQO9K-p07S7aaDAX-owWnVC6O1dsVn-gwfXS5fjeQUf5U1bTCShiycgwVkziq7taqcAXr2md21V5imVVLS581-q22hILUI1WcsRPv7zQk5VpUfptj5l6u1_33ZOqyWcSRpzQS_ATU93ISVb9MF6_DUGn8MJQ6_teie9KxteC2bLknPKjC5xlnV43Ni4ahVd9LG-nPXiqmJY0Mil0UxKsgFEi_hWzWr7tqCx6vLfvvGK2kUPMkYm3gWQPgsErG2kQJPuKGBormJjAllQEVEZZQksVF5nJs8N5hpTXMb1vmBMAmLI8W2YX44GuodIFwFJlBpasLEhBEzecRTaUMIKbjgXPkNaH6daybLGeNIdfGSOazh88xKIkNJZKUkGnBS7Xibzdf4Y20dT75aVx56A_a_RJeVF3CcUSQeteg3TnZ_33UESzf9-07Wue3e7cEyxVIVV6O3D_OTYqoPYFG-T57HxaHTsk_8mc6D
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Asynchronous+Optimization+Over+Graphs%3A+Linear+Convergence+Under+Error+Bound+Conditions&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Cannelli%2C+Loris&rft.au=Facchinei%2C+Francisco&rft.au=Scutari%2C+Gesualdo&rft.au=Kungurtsev%2C+Vyacheslav&rft.date=2021-10-01&rft.issn=0018-9286&rft.eissn=1558-2523&rft.volume=66&rft.issue=10&rft.spage=4604&rft.epage=4619&rft_id=info:doi/10.1109%2FTAC.2020.3033490&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TAC_2020_3033490
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon