Random Coordinate Descent Algorithms for Multi-Agent Convex Optimization Over Networks

In this paper, we develop randomized block-coordinate descent methods for minimizing multi-agent convex optimization problems with linearly coupled constraints over networks and prove that they obtain in expectation an ε accurate solution in at most O(1/λ 2 (Q)ϵ) iterations, where λ 2 (Q) is the sec...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on automatic control Ročník 58; číslo 8; s. 2001 - 2012
Hlavný autor:	Necoara, Ion
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	IEEE 01.08.2013
Predmet:	Algorithm design and analysis Convergence Convex functions Distributed control Eigenvalues and eigenfunctions Linear programming linearly coupled constraints Optimization random coordinate descent rate of convergence resource allocation problems smooth convex optimization Vectors
ISSN:	0018-9286, 1558-2523
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	In this paper, we develop randomized block-coordinate descent methods for minimizing multi-agent convex optimization problems with linearly coupled constraints over networks and prove that they obtain in expectation an ε accurate solution in at most O(1/λ 2 (Q)ϵ) iterations, where λ 2 (Q) is the second smallest eigenvalue of a matrix Q that is defined in terms of the probabilities and the number of blocks. However, the computational complexity per iteration of our methods is much simpler than the one of a method based on full gradient information and each iteration can be computed in a completely distributed way. We focus on how to choose the probabilities to make these randomized algorithms to converge as fast as possible and we arrive at solving a sparse SDP. Analysis for rate of convergence in probability is also provided. For strongly convex functions our distributed algorithms converge linearly. We also extend the main algorithm to a more general random coordinate descent method and to problems with more general linearly coupled constraints. Preliminary numerical tests confirm that on very large optimization problems our method is much more numerically efficient than methods based on full gradient.
AbstractList	In this paper, we develop randomized block-coordinate descent methods for minimizing multi-agent convex optimization problems with linearly coupled constraints over networks and prove that they obtain in expectation an ε accurate solution in at most O(1/λ 2 (Q)ϵ) iterations, where λ 2 (Q) is the second smallest eigenvalue of a matrix Q that is defined in terms of the probabilities and the number of blocks. However, the computational complexity per iteration of our methods is much simpler than the one of a method based on full gradient information and each iteration can be computed in a completely distributed way. We focus on how to choose the probabilities to make these randomized algorithms to converge as fast as possible and we arrive at solving a sparse SDP. Analysis for rate of convergence in probability is also provided. For strongly convex functions our distributed algorithms converge linearly. We also extend the main algorithm to a more general random coordinate descent method and to problems with more general linearly coupled constraints. Preliminary numerical tests confirm that on very large optimization problems our method is much more numerically efficient than methods based on full gradient.
Author	Necoara, Ion
Author_xml	– sequence: 1 givenname: Ion surname: Necoara fullname: Necoara, Ion email: ion.necoara@acse.pub.ro organization: Dept. of Autom. Control & Syst. Eng., Univ. Politeh. Bucharest, Bucharest, Romania
BookMark	eNp9kMtOwzAQRS1UJNrCHomNfyDFj_iRZRSeUqESKmwjN50UQxJXtimPryelFQsWrEajuWd0dUZo0LkOEDqlZEIpyc7neTFhhPIJY4IQRQ_QkAqhEyYYH6AhIVQnGdPyCI1CeOlXmaZ0iJ4eTLd0LS6c80vbmQj4AkIFXcR5s3Lexuc24Np5fPfWRJvkq-2pcN0GPvBsHW1rv0y0rsOzDXh8D_Hd-ddwjA5r0wQ42c8xery6nBc3yXR2fVvk06RikscEtMiUURXnRi2UMDVjlSI1CGrYkpGULPqiItVEsAUTStWZlmldARjDTSY1HyO5-1t5F4KHuqxs_OkTvbFNSUm5tVP2dsqtnXJvpwfJH3DtbWv853_I2Q6xAPAbl6miSnD-DfQ_cgA
CODEN	IETAA9
CitedBy_id	crossref_primary_10_1137_22M148700X crossref_primary_10_1016_j_sysconle_2015_06_006 crossref_primary_10_1049_gtd2_12142 crossref_primary_10_1080_10556788_2020_1750013 crossref_primary_10_1007_s11431_018_9451_4 crossref_primary_10_1007_s10898_014_0151_9 crossref_primary_10_1109_TAC_2016_2615066 crossref_primary_10_1109_TNSM_2025_3561269 crossref_primary_10_1109_TCNS_2019_2925267 crossref_primary_10_1109_TAC_2024_3394349 crossref_primary_10_1109_LCSYS_2020_2976311 crossref_primary_10_1016_j_neucom_2019_11_086 crossref_primary_10_1109_TSG_2020_2989828 crossref_primary_10_1109_TSMC_2020_3005169 crossref_primary_10_1109_TSP_2021_3114978 crossref_primary_10_1109_TCYB_2022_3178929 crossref_primary_10_1109_ACCESS_2018_2888855 crossref_primary_10_1109_TAC_2020_3041248 crossref_primary_10_1007_s10957_016_1058_z crossref_primary_10_1137_130950288 crossref_primary_10_1007_s10915_025_02992_0 crossref_primary_10_1093_imanum_draa018 crossref_primary_10_1109_TNSE_2023_3248267 crossref_primary_10_1109_TAC_2016_2604373 crossref_primary_10_1109_TAC_2016_2607023 crossref_primary_10_1109_LCSYS_2021_3084531 crossref_primary_10_1287_moor_2023_0044 crossref_primary_10_1109_TAC_2015_2390551 crossref_primary_10_1016_j_ifacol_2022_07_244 crossref_primary_10_1016_j_eswa_2022_118099 crossref_primary_10_1109_TAC_2017_2673240 crossref_primary_10_1016_j_engappai_2020_104115 crossref_primary_10_1016_j_sysconle_2016_11_006 crossref_primary_10_1016_j_automatica_2023_111339 crossref_primary_10_1007_s10107_020_01599_7 crossref_primary_10_1007_s10287_018_0303_3 crossref_primary_10_1109_TSMC_2024_3349407 crossref_primary_10_1109_TSMC_2021_3112691 crossref_primary_10_1016_j_automatica_2021_109739 crossref_primary_10_1007_s10589_019_00082_0 crossref_primary_10_1109_MCS_2014_2308672 crossref_primary_10_1016_j_automatica_2019_02_003 crossref_primary_10_1137_20M1328014 crossref_primary_10_1007_s10589_016_9824_2 crossref_primary_10_1109_TAC_2020_3027647 crossref_primary_10_1007_s40747_022_00785_8 crossref_primary_10_1109_TCNS_2017_2774010 crossref_primary_10_1016_j_automatica_2023_111051
Cites_doi	10.1007/s10589-009-9251-8 10.2307/2296432 10.1109/TSP.2009.2039825 10.1137/090780705 10.1137/100802001 10.1017/CBO9780511752940.002 10.1007/s10107-007-0170-0 10.1007/978-1-4419-8853-9 10.1007/s10107-012-0614-z 10.1110/ps.0242703 10.1109/32.92917 10.1007/978-1-4613-0163-9 10.1007/s10107-003-0471-x 10.3934/ipi.2009.3.487 10.1109/12.24272 10.1007/s10957-006-9080-1 10.1007/s10957-008-9458-3
ContentType	Journal Article
DBID	97E RIA RIE AAYXX CITATION
DOI	10.1109/TAC.2013.2250071
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) (UW System Shared) CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
Database_xml	– sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-2523
EndPage	2012
ExternalDocumentID	10_1109_TAC_2013_2250071 6471753
Genre	orig-research
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 VJK ~02 AAYXX CITATION
ID	FETCH-LOGICAL-c263t-e8597a7c33a7b75af22c70fe51a2d2040b001548052b2577f9864fceeaa3a9683
IEDL.DBID	RIE
ISICitedReferencesCount	85
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000322364300009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9286
IngestDate	Tue Nov 18 22:41:40 EST 2025 Sat Nov 29 02:26:00 EST 2025 Wed Aug 27 02:48:46 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	8
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c263t-e8597a7c33a7b75af22c70fe51a2d2040b001548052b2577f9864fceeaa3a9683
PageCount	12
ParticipantIDs	ieee_primary_6471753 crossref_primary_10_1109_TAC_2013_2250071 crossref_citationtrail_10_1109_TAC_2013_2250071
PublicationCentury	2000
PublicationDate	2013-08-01
PublicationDateYYYYMMDD	2013-08-01
PublicationDate_xml	– month: 08 year: 2013 text: 2013-08-01 day: 01
PublicationDecade	2010
PublicationTitle	IEEE transactions on automatic control
PublicationTitleAbbrev	TAC
PublicationYear	2013
Publisher	IEEE
Publisher_xml	– name: IEEE
References	ref12 godsil (ref5) 2001 ref14 qin (ref19) 2010 nesterov (ref15) 2004 knuth (ref10) 1981 ho (ref7) 1980; 1 ref2 ref17 ref16 ref18 wei (ref22) 2011 chang (ref3) 2008; 9 bertsekas (ref1) 1999 ref23 ref26 ref25 ref20 wright (ref21) 2010 li (ref11) 2009; 3 necoara (ref13) 2011 ref8 ref9 ref4 yuan (ref24) 2011 ref6
References_xml	– year: 1999 ident: ref1 publication-title: Nonlinear Programming – ident: ref25 doi: 10.1007/s10589-009-9251-8 – year: 1981 ident: ref10 publication-title: The art of computer programming – ident: ref6 doi: 10.2307/2296432 – ident: ref18 doi: 10.1109/TSP.2009.2039825 – volume: 9 start-page: 1369 year: 2008 ident: ref3 article-title: Coordinate descent method for large-scale <tex Notation="TeX">$l_{2}$</tex>-loss linear support vector machines publication-title: J Mach Learn Res – ident: ref4 doi: 10.1137/090780705 – ident: ref14 doi: 10.1137/100802001 – ident: ref8 doi: 10.1017/CBO9780511752940.002 – year: 2011 ident: ref13 publication-title: ?A random coordinate descent method on large optimization problems with linear constraints ? – ident: ref12 doi: 10.1007/s10107-007-0170-0 – year: 2011 ident: ref22 publication-title: ?A distributed newton method for network utility maximization ? – year: 2004 ident: ref15 publication-title: Introductory Lectures on Convex Optimization A Basic Course doi: 10.1007/978-1-4419-8853-9 – ident: ref16 doi: 10.1007/s10107-012-0614-z – ident: ref2 doi: 10.1110/ps.0242703 – ident: ref20 doi: 10.1109/32.92917 – year: 2001 ident: ref5 publication-title: Algebraic Graph Theory doi: 10.1007/978-1-4613-0163-9 – ident: ref26 doi: 10.1007/s10107-003-0471-x – volume: 3 start-page: 487 year: 2009 ident: ref11 article-title: Coordinate descent optimization for <tex Notation="TeX">$l_{1}$</tex> minimization with application to compressed sensing; A greedy algorithm publication-title: Inverse Problems Imag doi: 10.3934/ipi.2009.3.487 – volume: 1 start-page: 51 year: 1980 ident: ref7 article-title: A class of center-free resource allocation algorithms publication-title: Large Scale Syst – year: 2010 ident: ref21 publication-title: ?Accelerated block coordinate relaxation for regularized optimization ? – ident: ref9 doi: 10.1109/12.24272 – ident: ref23 doi: 10.1007/s10957-006-9080-1 – year: 2011 ident: ref24 publication-title: ?Recent advances of large-scale linear classification ? – ident: ref17 doi: 10.1007/s10957-008-9458-3 – year: 2010 ident: ref19 publication-title: ?Efficient block-coordinate descent algorithms for the group lasso ?
SSID	ssj0016441
Score	2.4178247
Snippet	In this paper, we develop randomized block-coordinate descent methods for minimizing multi-agent convex optimization problems with linearly coupled constraints...
SourceID	crossref ieee
SourceType	Enrichment Source Index Database Publisher
StartPage	2001
SubjectTerms	Algorithm design and analysis Convergence Convex functions Distributed control Eigenvalues and eigenfunctions Linear programming linearly coupled constraints Optimization random coordinate descent rate of convergence resource allocation problems smooth convex optimization Vectors
Title	Random Coordinate Descent Algorithms for Multi-Agent Convex Optimization Over Networks
URI	https://ieeexplore.ieee.org/document/6471753
Volume	58
WOSCitedRecordID	wos000322364300009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared) customDbUrl: eissn: 1558-2523 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0016441 issn: 0018-9286 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LS8MwGA_b8KAHX1OcL3LwItitTdomPY7p8LSJTNmtJGmqg62VrRv--X5Ja5kggpdSylcov3zPfi-EbqTgnqfcwFGJ7zu-govkKnJEKHzflZrIhNtlE2w04tNp9NRAd3UvjNbaFp_prrm1ufwkV2vzq6wXgiYF97qJmoyFZa9WnTEwdr3UuiDAhNcpSTfqTfoDU8NFu8C7xqT-MEFbO1WsSRke_O9jDtF-5TrifnnWR6ihs2O0tzVQsI1en0WW5As8yCGmnGXgR-L7cl4T7s_f8uWseF-sMPip2DbeOn3TWAXU2UZ_4jFoj0XVlonHwOJ4VNaIr07Qy_BhMnh0qs0JjiIhLRzNIU4QTFEqmGSBSAlRzE114AmSEJDbKlZxAyJBZllqhrSnYC-FoCIKOT1FrSzP9BnCRGmfUy1FRCFyiiIhXUG9NOTKc9M0UB3U-wYzVtVYcbPdYh7b8MKNYoA_NvDHFfwddFu_8VGO1PiDtm2Qr-kq0M9_f3yBdondVmHq8y5Rq1iu9RXaUZtitlpeW375AonHvYI
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JS8NAFH5oFdSDWxV35-BFMHYyk2VyLNWiqK1IFW9hMplooU2kjeLP900SQwURvIQQXiB889a8DeAkksK2FXUtFTuO5Si8REIFlvSk49BIsygWxbIJv9cTz8_B_Ryc1b0wWuui-Eyfm9silx9n6t38Kmt5qEnRvZ6HBddxGC27teqcgbHspd5FEWaiTkrSoDVod0wVFz9H7jVG9YcRmtmqUhiV7tr_PmcdVivnkbTL096AOZ1uwsrMSMEmPD3INM7GpJNhVDlM0ZMkF-XEJtIevWSTYf46nhL0VEnRemu1TWsVUqcf-pP0UX-Mq8ZM0kcmJ72ySny6BY_dy0Hnyqp2J1iKeTy3tMBIQfqKc-lHvisTxpRPE-3aksUMJbeKVqjLIpRaPzFj2hO0mFJyGXiCb0MjzVK9A4Qp7QiuIxlwjJ2CQEZUcjvxhLJpkrhqF1rfYIaqGixu9luMwiLAoEGI8IcG_rCCfxdO6zfeyqEaf9A2DfI1XQX63u-Pj2HpanB3G95e9272YZkVuytMtd4BNPLJuz6ERfWRD6eTo4J3vgCSrcDJ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Random+Coordinate+Descent+Algorithms+for+Multi-Agent+Convex+Optimization+Over+Networks&rft.jtitle=IEEE+transactions+on+automatic+control&rft.au=Necoara%2C+Ion&rft.date=2013-08-01&rft.pub=IEEE&rft.issn=0018-9286&rft.volume=58&rft.issue=8&rft.spage=2001&rft.epage=2012&rft_id=info:doi/10.1109%2FTAC.2013.2250071&rft.externalDocID=6471753
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9286&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9286&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9286&client=summon