Double Averaging and Gradient Projection: Convergence Guarantees for Decentralized Constrained Optimization
We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm,...
Uloženo v:
| Vydáno v: | IEEE transactions on automatic control Ročník 70; číslo 5; s. 3433 - 3440 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
IEEE
01.05.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 0018-9286, 1558-2523, 1558-2523 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm, called double averaging and gradient projection (DAGP). We achieve global optimality through a novel distributed tracking technique we call distributed null projection. Further, we show that DAGP can be used to solve unconstrained problems with nondifferentiable objective terms with a problem reduction scheme. Assuming only smoothness of the objective terms, we study the convergence of DAGP and establish sublinear rates of convergence in terms of feasibility, consensus, and optimality, with no extra assumption (e.g., strong convexity). For the analysis, we forego the difficulties of selecting Lyapunov functions by proposing a new methodology of convergence analysis, which we refer to as aggregate lower-bounding. To demonstrate the generality of this method, we also provide an alternative convergence proof for the standard gradient descent algorithm with smooth functions. |
|---|---|
| AbstractList | We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm, called double averaging and gradient projection (DAGP). We achieve global optimality through a novel distributed tracking technique we call distributed null projection. Further, we show that DAGP can be used to solve unconstrained problems with nondifferentiable objective terms with a problem reduction scheme. Assuming only smoothness of the objective terms, we study the convergence of DAGP and establish sublinear rates of convergence in terms of feasibility, consensus, and optimality, with no extra assumption (e.g., strong convexity). For the analysis, we forego the difficulties of selecting Lyapunov functions by proposing a new methodology of convergence analysis, which we refer to as aggregate lower-bounding. To demonstrate the generality of this method, we also provide an alternative convergence proof for the standard gradient descent algorithm with smooth functions. We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm, called DAGP (Double Averaging and Gradient Projection). We achieve global optimality through a novel distributed tracking technique we call distributed null projection. Further, we show that DAGP can be used to solve unconstrained problems with non-differentiable objective terms with a problem reduction scheme. Assuming only smoothness of the objective terms, we study the convergence of DAGP and establish sub-linear rates of convergence in terms of feasibility, consensus, and optimality, with no extra assumption (e.g. strong convexity). For the analysis, we forego the difficulties of selecting Lyapunov functions by proposing a new methodology of convergence analysis, which we refer to as aggregate lower-bounding. To demonstrate the generality of this method, we also provide an alternative convergence proof for the standard gradient descent algorithm with smooth functions. |
| Author | Shahriari-Mehr, Firooz Panahi, Ashkan |
| Author_xml | – sequence: 1 givenname: Firooz orcidid: 0000-0003-2374-2341 surname: Shahriari-Mehr fullname: Shahriari-Mehr, Firooz email: firooz@chalmers.se organization: Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Gothenburg, Sweden – sequence: 2 givenname: Ashkan orcidid: 0000-0003-2085-7127 surname: Panahi fullname: Panahi, Ashkan email: ashkan.panahi@chalmers.se organization: Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Gothenburg, Sweden |
| BackLink | https://gup.ub.gu.se/publication/350004$$DView record from Swedish Publication Index (Göteborgs universitet) https://research.chalmers.se/publication/544526$$DView record from Swedish Publication Index (Chalmers tekniska högskola) |
| BookMark | eNp1kU1v1DAQhiNUJLaFOwcOkThnO44_YnNbbWGLVKlIlLNlx-M0y6692AmI_nocbYW4cJoPPfNIo_eyuggxYFW9JbAmBNT1w2a7bqFla8pb4IS-qFaEc9m0vKUX1QqAyEa1UryqLnPel1EwRlbV95s42wPWm5-YzDCGoTbB1btk3Ihhqr-kuMd-GmP4UG9jKNCAocd6N5tkwoSYax9TfYN9oZM5jE_oFjCXYQylvz9N43F8MovidfXSm0PGN8_1qvr26ePD9ra5u9993m7ump5RMjXOIIhOCAtSKapU50lnwTLpvXQ94WCNd6TljAuiWmGlM-Ba3xIurBLI6VX19ezNv_A0W31K49Gk3zqaUSfMaFL_qPtHczhiyjqjtt5zgeC0cT1oZqXUEqTTHBwwYoBKqoq1-a91mE-6rIZ5sVEOAKzw78_8KcUfM-ZJ7-OcQnlcU6JYJzrJRaHgTPUp5pzQ__US0EuwugSrl2D1c7Dl5N35ZETEf3BJQAhC_wBl0KI4 |
| CODEN | IETAA9 |
| Cites_doi | 10.1109/TSIPN.2016.2524588 10.1109/TSP.2009.2038417 10.1109/TAC.2022.3192316 10.1109/TIT.2006.874516 10.1109/TAC.2008.2009515 10.1016/j.automatica.2009.10.021 10.1137/S003614450037906X 10.1109/JPROC.2018.2817461 10.1109/LCSYS.2018.2834316 10.1007/s10107-018-01357-w 10.1016/j.arcontrol.2019.05.006 10.1109/MSP.2020.2975210 10.1137/16M1084316 10.1109/JSTSP.2013.2247023 10.1109/SFCS.2003.1238221 10.1137/14096668X 10.1109/TAC.2010.2041686 10.1561/2400000003 10.1109/TIT.2012.2191450 10.1007/978-1-4615-0363-7 10.1109/TAC.2016.2615066 10.1007/s10957-010-9737-7 10.1109/CDC.2012.6426375 10.1109/TAC.2010.2091295 10.1109/TAC.2017.2672698 10.1109/TCNS.2023.3338236 10.1109/JPROC.2020.3024266 10.1109/TAC.2017.2737582 10.1109/TAC.2012.2209984 10.1007/s10107-011-0467-x 10.1109/MSP.2020.2974267 10.1109/TAC.2014.2364096 10.1109/TAC.2020.2972824 10.1007/BF00994018 10.3166/EJC.18.539-557 10.1007/s00026-005-0237-z 10.1109/CDC45484.2021.9683355 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D ADTPV AOWAS F1U F1S |
| DOI | 10.1109/TAC.2024.3520513 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional SwePub SwePub Articles SWEPUB Göteborgs universitet SWEPUB Chalmers tekniska högskola |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-2523 |
| EndPage | 3440 |
| ExternalDocumentID | oai_research_chalmers_se_bff56e0d_adc0_4b88_808d_50d041a03839 oai_gup_ub_gu_se_350004 10_1109_TAC_2024_3520513 10810661 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Wallenberg AI – fundername: Autonomous Systems – fundername: Software Program – fundername: Alice Wallenberg Foundation |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 VJK ~02 AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D ADTPV AOWAS F1U F1S |
| ID | FETCH-LOGICAL-c431t-dae06766b08993997f17b0b48ff8dc150bafd1254561926b8da0d2f2156b96e53 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001477383000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0018-9286 1558-2523 |
| IngestDate | Wed Nov 05 04:11:00 EST 2025 Tue Nov 04 16:25:48 EST 2025 Mon Jun 30 10:16:36 EDT 2025 Sat Nov 29 07:58:03 EST 2025 Wed Aug 27 02:03:31 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c431t-dae06766b08993997f17b0b48ff8dc150bafd1254561926b8da0d2f2156b96e53 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-2374-2341 0000-0003-2085-7127 |
| PQID | 3194767856 |
| PQPubID | 85475 |
| PageCount | 8 |
| ParticipantIDs | swepub_primary_oai_research_chalmers_se_bff56e0d_adc0_4b88_808d_50d041a03839 swepub_primary_oai_gup_ub_gu_se_350004 ieee_primary_10810661 crossref_primary_10_1109_TAC_2024_3520513 proquest_journals_3194767856 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-05-01 |
| PublicationDateYYYYMMDD | 2025-05-01 |
| PublicationDate_xml | – month: 05 year: 2025 text: 2025-05-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on automatic control |
| PublicationTitleAbbrev | TAC |
| PublicationYear | 2025 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref35 ref12 ref15 ref37 ref14 ref36 Forero (ref38) 2010; 11 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 ref39 ref16 ref19 Wang (ref34) 2015 Shahriari-Mehr (ref43) 2023 Tsianos (ref18) 2012; 25 ref24 ref23 ref26 ref25 ref20 ref42 ref41 ref22 ref21 ref28 ref27 Bishop (ref6) 2006 ref29 Calafiore (ref40) 2014 ref8 ref7 ref9 ref4 ref3 ref5 Ram (ref17) 2010; 147 Beck (ref44) 2010 |
| References_xml | – ident: ref25 doi: 10.1109/TSIPN.2016.2524588 – ident: ref2 doi: 10.1109/TSP.2009.2038417 – ident: ref19 doi: 10.1109/TAC.2022.3192316 – volume: 25 start-page: 1943 volume-title: Proc. Int. Conf. Adv. Neural Inf. Process. Syst. year: 2012 ident: ref18 article-title: Communication/computation tradeoffs in consensus-based distributed optimization – ident: ref11 doi: 10.1109/TIT.2006.874516 – ident: ref20 doi: 10.1109/TAC.2008.2009515 – ident: ref22 doi: 10.1016/j.automatica.2009.10.021 – ident: ref39 doi: 10.1137/S003614450037906X – ident: ref7 doi: 10.1109/JPROC.2018.2817461 – ident: ref14 doi: 10.1109/LCSYS.2018.2834316 – ident: ref31 doi: 10.1007/s10107-018-01357-w – ident: ref8 doi: 10.1016/j.arcontrol.2019.05.006 – ident: ref10 doi: 10.1109/MSP.2020.2975210 – ident: ref23 doi: 10.1137/16M1084316 – ident: ref37 doi: 10.1109/JSTSP.2013.2247023 – ident: ref27 doi: 10.1109/SFCS.2003.1238221 – ident: ref24 doi: 10.1137/14096668X – ident: ref36 doi: 10.1109/TAC.2010.2041686 – ident: ref41 doi: 10.1561/2400000003 – ident: ref3 doi: 10.1109/TIT.2012.2191450 – ident: ref4 doi: 10.1007/978-1-4615-0363-7 – ident: ref15 doi: 10.1109/TAC.2016.2615066 – volume: 147 start-page: 516 issue: 3 year: 2010 ident: ref17 article-title: Distributed stochastic subgradient projection algorithms for convex optimization publication-title: J. Optim. Theory Appl. doi: 10.1007/s10957-010-9737-7 – ident: ref28 doi: 10.1109/CDC.2012.6426375 – ident: ref21 doi: 10.1109/TAC.2010.2091295 – volume-title: Pattern Recognition and Machine Learning year: 2006 ident: ref6 – ident: ref30 doi: 10.1109/TAC.2017.2672698 – ident: ref33 doi: 10.1109/TCNS.2023.3338236 – volume: 11 start-page: 1663 issue: 55 year: 2010 ident: ref38 article-title: Consensus-based distributed support vector machines publication-title: J. Mach. Learn. Res. – ident: ref12 doi: 10.1109/JPROC.2020.3024266 – ident: ref32 doi: 10.1109/TAC.2017.2737582 – volume-title: Convex Optimization in Signal Processing and Communications year: 2010 ident: ref44 article-title: Gradient-based algorithms with applications to signal-recovery problems – ident: ref16 doi: 10.1109/TAC.2012.2209984 – ident: ref35 doi: 10.1007/s10107-011-0467-x – ident: ref9 doi: 10.1109/MSP.2020.2974267 – ident: ref29 doi: 10.1109/TAC.2014.2364096 – ident: ref13 doi: 10.1109/TAC.2020.2972824 – ident: ref5 doi: 10.1007/BF00994018 – volume-title: Optimization Models, Ser. Control Systems and Optimization Series year: 2014 ident: ref40 – ident: ref26 doi: 10.3166/EJC.18.539-557 – year: 2015 ident: ref34 article-title: Random multi-constraint projection: Stochastic gradient methods for convex optimization with many constraints – ident: ref42 doi: 10.1007/s00026-005-0237-z – ident: ref1 doi: 10.1109/CDC45484.2021.9683355 – year: 2023 ident: ref43 article-title: Double averaging and gradient projection: Convergence guarantees for decentralized constrained optimization |
| SSID | ssj0016441 |
| Score | 2.477806 |
| Snippet | We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to... |
| SourceID | swepub proquest crossref ieee |
| SourceType | Open Access Repository Aggregation Database Index Database Publisher |
| StartPage | 3433 |
| SubjectTerms | Algorithms Communication networks Computer architecture Computer Sciences Constrained optimization Constraints Convergence convergence analysis Convex functions convex optimization Convexity Datavetenskap (datalogi) Directed graphs distributed optimization Liapunov functions Linear programming Lyapunov methods Networked, Parallel and Distributed Computing Nätverks-, parallell- och distribuerad beräkning Optimization Optimization methods Smoothness Vectors |
| Title | Double Averaging and Gradient Projection: Convergence Guarantees for Decentralized Constrained Optimization |
| URI | https://ieeexplore.ieee.org/document/10810661 https://www.proquest.com/docview/3194767856 https://gup.ub.gu.se/publication/350004 https://research.chalmers.se/publication/544526 |
| Volume | 70 |
| WOSCitedRecordID | wos001477383000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 1558-2523 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NTxQxFG-QcNCDimIYQdMDMeEw2HZn-uFtswocCHJAwq3pxysQZSE7ux7863ntDJsl0YO3ZtJmmvde-36v74uQPeH4qHFoqcYkUt1Aq2vPoqrDSBrNQxKmdIm4OFGnp_ry0pwNyeolFwYASvAZHORh8eXHu7DIT2V4wjVaMNnYeaaU6pO1li6DrNj7axdPsNBLnyQzn8_HE7QERXOAaAOFcPREB5WmKk_x5WrN0KJnDl_95w5fk5cDoKTjXgI2yRpM35AXK2UG35KfiJL9L6BjFNvSlIi6aaRHsxLtNadn_WMMMugLneQg9JKPCTRLT6Y7dBSRLf0KQyDnzR-IeWJX2kvg-DteO7dDPucW-XH47XxyXA9NFuqA2GFeRweosKT02f-HaEUlrjzzjU5Jx4Bw0bsUEQVloGWE9Do6FkVCpCC9kdCO3pH16d0Utgl1SQdIaN8ZHnNjSu0RHvHIA1cSuICK7D-S3d73tTRssUGYscgim1lkBxZVZCtTd2VeT9iK7D4yyg6nrbN4jTQKtW4rK_KpZ95yYS6ffbW4t_jpamE7wB9kHFuRk79MHAosXdtwXbrXdHmBT6mVwKJ1MTDbeK2tZjralkXWcMfQxjfv_7HdHfJc5IbBJUJyl6zPZwv4QDbC7_lNN_tYBPgBNAPwDg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LbxMxELZQQQIOPItYKOADQuKwxXa8XptbFChFhNBDQL1ZfozbCkirbMKBX8_Yu41SCQ7crJWttWbGnm88L0JeCsdH0qGlGpNItYRG157Ftg4jZTQPSZjSJeLbtJ3N9PGxORqS1UsuDACU4DPYz8Piy4_nYZ2fyvCEa7RgsrFzvZFS8D5da-M0yKq9v3jxDAu98Uoy82Y-nqAtKOQ-4g0Uw9EVLVTaqlxFmNtVQ4umObj7n3u8R-4MkJKOexm4T67B4gG5vVVo8CH5jjjZ_wA6RsEtbYmoW0T6YVnivVb0qH-OQRa9pZMchl4yMoFm-cmUh44itqXvYAjlPPsNMU_sSoMJHH_Bi-fnkNG5S74evJ9PDuuhzUIdED2s6ugAVZZSPnsAEa-0ibeeealT0jEgYPQuRcRBGWoZobyOjkWRECsobxQ0o0dkZ3G-gMeEuqQDJLTwDI-5NaX2CJB45IG3CriAiry-JLu96Ktp2GKFMGORRTazyA4sqshupu7WvJ6wFdm7ZJQdzltn8SKRLerdRlXkVc-8zcJcQPtkfWHx08nadoA_yEi2ItO_TBxKLJ3acFr613R5gU-pUcCidTEwK73WVjMdbcMik9wxtPLNk39s9wW5eTj_PLXTj7NPT8ktkdsHl3jJPbKzWq7hGbkRfq3OuuXzIsx_AM-S81U |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Double+Averaging+and+Gradient+Projection%3A+Convergence+Guarantees+for+Decentralized+Constrained+Optimization&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Shahriari+Mehr%2C+Firooz&rft.au=Panahi%2C+Ashkan&rft.date=2025-05-01&rft.issn=1558-2523&rft.volume=70&rft.issue=5&rft.spage=3433&rft_id=info:doi/10.1109%2FTAC.2024.3520513&rft.externalDocID=oai_research_chalmers_se_bff56e0d_adc0_4b88_808d_50d041a03839 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon |