Parallel Selective Algorithms for Nonconvex Big Data Optimization

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on signal processing Ročník 63; číslo 7; s. 1874 - 1889
Hlavní autori:	Facchinei, Francisco, Scutari, Gesualdo, Sagratella, Simone
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	IEEE 01.04.2015
Predmet:	Approximation methods Convergence distributed methods Jacobi method Jacobian matrices LASSO Optimization Parallel optimization Signal processing algorithms sparse solution variables selection Vectors
ISSN:	1053-587X, 1941-0476
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
AbstractList	We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities "in between" with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
Author	Scutari, Gesualdo Sagratella, Simone Facchinei, Francisco
Author_xml	– sequence: 1 givenname: Francisco surname: Facchinei fullname: Facchinei, Francisco email: facchinei@dis.uniroma1.it organization: Dept. of Comput., Control, & Manage. Eng., Univ. of Rome La Sapienza, Rome, Italy – sequence: 2 givenname: Gesualdo surname: Scutari fullname: Scutari, Gesualdo email: gesualdo@buffalo.edu organization: Dept. of Electr. Eng., State Univ. of New York at Buffalo, Buffalo, NY, USA – sequence: 3 givenname: Simone surname: Sagratella fullname: Sagratella, Simone email: sagratella@dis.uniroma1.it organization: Dept. of Comput., Control, & Manage. Eng., Univ. of Rome La Sapienza, Rome, Italy
BookMark	eNp9kD1PwzAQQC1UJNrCjsTiP5Dij9ixx1A-pYpWapHYIie5FCM3rhyrAn49Ka0YGJjulnf39EZo0PoWELqkZEIp0der5WLCCBUTxrVWQp2gIdUpTUiayUG_E8ETobLXMzTqundCaJpqOUT5wgTjHDi8BAdVtDvAuVv7YOPbpsOND_jZt5Vvd_CBb-wa35po8Hwb7cZ-mWh9e45OG-M6uDjOMXq5v1tNH5PZ_OFpms-SihMRE8M5E5rJUpUZYUpzw7SpmayYrGlaN1pnAhQpjYFaKFBl70205JwbyjIKfIzI4W4VfNcFaIptsBsTPgtKin2Cok9Q7BMUxwQ9Iv8glY0_0jEY6_4Drw6gBYDfP723zqjk30Pnao4
CODEN	ITPRED
CitedBy_id	crossref_primary_10_1109_TCST_2018_2827990 crossref_primary_10_1109_ACCESS_2019_2926616 crossref_primary_10_22430_22565337_1344 crossref_primary_10_1007_s10107_018_01357_w crossref_primary_10_1007_s41870_024_01801_5 crossref_primary_10_1631_FITEE_1800566 crossref_primary_10_1109_LSP_2018_2859596 crossref_primary_10_1109_TSP_2016_2637317 crossref_primary_10_1080_1206212X_2017_1417768 crossref_primary_10_1109_OJSP_2025_3528875 crossref_primary_10_1109_TAC_2022_3183036 crossref_primary_10_1007_s10898_019_00792_z crossref_primary_10_1007_s10957_023_02328_8 crossref_primary_10_1109_TSP_2018_2868040 crossref_primary_10_1016_j_sigpro_2025_110077 crossref_primary_10_1007_s13369_016_2405_y crossref_primary_10_1016_j_sigpro_2019_107433 crossref_primary_10_1109_TSIPN_2016_2561703 crossref_primary_10_1016_j_future_2018_06_008 crossref_primary_10_3390_fractalfract6120709 crossref_primary_10_1007_s12559_016_9439_z crossref_primary_10_1109_TAC_2020_3033490 crossref_primary_10_1007_s10107_022_01836_1 crossref_primary_10_1109_JSTSP_2018_2877584 crossref_primary_10_1109_TAC_2016_2607023 crossref_primary_10_1007_s10107_019_01408_w crossref_primary_10_1007_s10589_018_00057_7 crossref_primary_10_1287_moor_2020_1079 crossref_primary_10_1360_SSI_2025_0112 crossref_primary_10_31548_dopovidi2019_03_010 crossref_primary_10_1109_TCI_2022_3148062 crossref_primary_10_1109_TSIPN_2015_2448520 crossref_primary_10_1111_wre_12255 crossref_primary_10_1109_TNNLS_2020_3027381 crossref_primary_10_1007_s10589_018_9984_3 crossref_primary_10_1016_j_sigpro_2022_108459 crossref_primary_10_1371_journal_pcbi_1005538 crossref_primary_10_1109_TSP_2018_2818081 crossref_primary_10_1007_s11042_018_6391_9 crossref_primary_10_1109_TSP_2020_3001906 crossref_primary_10_1016_j_cviu_2022_103539 crossref_primary_10_1109_TSP_2020_3031056 crossref_primary_10_1109_TCNS_2017_2774010 crossref_primary_10_1016_j_ins_2018_10_005 crossref_primary_10_1109_TSP_2020_3033354 crossref_primary_10_1109_TSP_2016_2544743 crossref_primary_10_1109_ACCESS_2019_2925429 crossref_primary_10_1007_s00186_022_00799_5 crossref_primary_10_1080_02331934_2021_1939336 crossref_primary_10_1109_TNNLS_2017_2785361 crossref_primary_10_1016_j_neunet_2016_04_007 crossref_primary_10_1007_s10107_016_1072_9 crossref_primary_10_1109_TII_2021_3062040 crossref_primary_10_1016_j_neunet_2017_04_004 crossref_primary_10_1155_2017_3610283 crossref_primary_10_1016_j_tre_2018_03_011 crossref_primary_10_1016_j_frl_2018_08_001 crossref_primary_10_1109_TSP_2016_2531627 crossref_primary_10_1109_TAC_2020_3008713 crossref_primary_10_1109_TSP_2017_2684748 crossref_primary_10_1109_TWC_2015_2489640 crossref_primary_10_1109_TSP_2015_2452219 crossref_primary_10_1137_19M1259973 crossref_primary_10_1109_TSIPN_2016_2524588 crossref_primary_10_1109_TSP_2016_2601299 crossref_primary_10_1109_TSP_2019_2959240 crossref_primary_10_1109_TGRS_2018_2870486 crossref_primary_10_1007_s10589_018_9987_0 crossref_primary_10_1109_MSP_2015_2481563 crossref_primary_10_1109_TSP_2020_2986363 crossref_primary_10_1007_s11227_019_03129_5 crossref_primary_10_1109_JIOT_2021_3064341 crossref_primary_10_1109_TSP_2015_2436357 crossref_primary_10_1093_nar_gkx1246 crossref_primary_10_1109_TSIPN_2023_3239698 crossref_primary_10_1109_TMI_2019_2934125 crossref_primary_10_1109_TAC_2020_3027647 crossref_primary_10_1109_TWC_2022_3160051
Cites_doi	10.1109/ICASSP.2014.6854999 10.1016/j.sigpro.2011.01.012 10.1137/S105262349427577X 10.1109/TSP.2013.2293126 10.1007/BF00935173 10.1111/j.1467-9868.2007.00627.x 10.1137/080716542 10.3934/ipi.2009.3.487 10.1007/s10107-012-0629-5 10.1111/j.2517-6161.1996.tb02080.x 10.1145/1553374.1553463 10.1111/j.1467-9868.2005.00532.x 10.1007/s10107-012-0530-2 10.1007/s12532-013-0051-x 10.1007/s10107-007-0170-0 10.1093/bioinformatics/btg308 10.1007/BF00934554 10.1007/978-3-642-20192-9 10.1016/j.jprocont.2012.12.012 10.1109/TSP.2009.2016892 10.1137/100802001 10.1109/TAC.1978.1101718 10.1137/120891009 10.1080/00207728108963798 10.1109/ACSSC.2013.6810364
ContentType	Journal Article
DBID	97E RIA RIE AAYXX CITATION
DOI	10.1109/TSP.2015.2399858
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1941-0476
EndPage	1889
ExternalDocumentID	10_1109_TSP_2015_2399858 7029716
Genre	orig-research
GrantInformation_xml	– fundername: MIUR project PLATINO (Grant Agreement – fundername: CAREER – fundername: USA National Science Foundation under Grants grantid: CMS 1218717
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK ACNCT AENEX AGQYO AGSQL AHBIQ AJQPL AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 3EH 53G 5VS AAYXX ABFSI ACKIV AETIX AI. AIBXA AKJIK ALLEH CITATION E.L H~9 ICLAB IFJZH VH1
ID	FETCH-LOGICAL-c305t-a3325926b8b702893a29ad26c26d14df9975e80baaed58e8b476096333a1271e3
IEDL.DBID	RIE
ISICitedReferencesCount	129
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000350880900020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1053-587X
IngestDate	Sat Nov 29 04:10:34 EST 2025 Tue Nov 18 22:35:49 EST 2025 Tue Aug 26 16:40:39 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	7
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c305t-a3325926b8b702893a29ad26c26d14df9975e80baaed58e8b476096333a1271e3
OpenAccessLink	http://hdl.handle.net/11573/771792
PageCount	16
ParticipantIDs	crossref_primary_10_1109_TSP_2015_2399858 crossref_citationtrail_10_1109_TSP_2015_2399858 ieee_primary_7029716
PublicationCentury	2000
PublicationDate	2015-April1, 2015-4-00
PublicationDateYYYYMMDD	2015-04-01
PublicationDate_xml	– month: 04 year: 2015 text: 2015-April1, day: 01
PublicationDecade	2010
PublicationTitle	IEEE transactions on signal processing
PublicationTitleAbbrev	TSP
PublicationYear	2015
Publisher	IEEE
Publisher_xml	– name: IEEE
References	ref13 ref34 ref12 ref37 ref36 ref14 ref31 ref30 ref11 ref32 ref10 b hlmann (ref15) 2011 ref1 ref39 ref38 bradley (ref23) 2011 (ref16) 2011 bertsekas (ref33) 1989 deng (ref40) 2014 bach (ref17) 2011 tibshirani (ref2) 1996 necoara (ref25) 2013 lu (ref26) 2013 facchinei (ref35) 2003 yuan (ref5) 2010; 11 ref22 bertsekas (ref41) 2011 ref21 dhillon (ref19) 2011 scherrer (ref20) 2012 ref28 ref29 ref8 li (ref18) 2009; 3 ref9 ref4 richt rik (ref27) 2012 ref3 necoara (ref24) 2011 byrd (ref6) 2013 fountoulakis (ref7) 2013
References_xml	– ident: ref1 doi: 10.1109/ICASSP.2014.6854999 – ident: ref4 doi: 10.1016/j.sigpro.2011.01.012 – ident: ref34 doi: 10.1137/S105262349427577X – ident: ref21 doi: 10.1109/TSP.2013.2293126 – ident: ref39 doi: 10.1007/BF00935173 – ident: ref30 doi: 10.1111/j.1467-9868.2007.00627.x – ident: ref11 doi: 10.1137/080716542 – volume: 3 start-page: 487 year: 2009 ident: ref18 article-title: Coordinate descent optimization for <formula formulatype="inline"><tex Notation="TeX">$\ell 1$</tex></formula> minimization with application to compressed sensing; a greedy algorithm publication-title: Inverse Probl Imag doi: 10.3934/ipi.2009.3.487 – start-page: 2160 year: 2011 ident: ref19 publication-title: Advances in Neural Information Processing Systems 24 – year: 2013 ident: ref26 publication-title: ?Randomized block coordinate non-monotone gradient method for a class of nonlinear programming ? – ident: ref9 doi: 10.1007/s10107-012-0629-5 – year: 2013 ident: ref6 publication-title: ?An inexact successive quadratic approximation method for convex L-1 regularized optimization ? – start-page: 267 year: 1996 ident: ref2 article-title: Regression shrinkage and selection via the lasso publication-title: J Roy Statist Soc Series B (Methodol ) doi: 10.1111/j.2517-6161.1996.tb02080.x – ident: ref31 doi: 10.1145/1553374.1553463 – ident: ref28 doi: 10.1111/j.1467-9868.2005.00532.x – ident: ref32 doi: 10.1007/s10107-012-0530-2 – volume: 11 start-page: 3183 year: 2010 ident: ref5 article-title: A comparison of optimization methods and software for large-scale l1-regularized linear classification publication-title: J Mach Learn Res – ident: ref3 doi: 10.1007/s12532-013-0051-x – ident: ref10 doi: 10.1007/s10107-007-0170-0 – year: 2014 ident: ref40 publication-title: ?Parallel multi-block ADMM with convergence ? – ident: ref29 doi: 10.1093/bioinformatics/btg308 – ident: ref37 doi: 10.1007/BF00934554 – year: 2013 ident: ref7 publication-title: ?A second-order method for strongly convex L1-regularization problems ? – year: 2011 ident: ref23 article-title: Parallel coordinate descent for l1-regularized loss minimization publication-title: Proc 28th Int Conf Mach Learn – start-page: 1 year: 2013 ident: ref25 publication-title: ?Distributed random coordinate descent method for composite minimization ? – year: 2011 ident: ref15 publication-title: Statistics for High-Dimensional Data doi: 10.1007/978-3-642-20192-9 – year: 2011 ident: ref17 publication-title: Foundations and Trends in Machine Learning – year: 2011 ident: ref16 publication-title: Optimization for Machine Learning – ident: ref8 doi: 10.1016/j.jprocont.2012.12.012 – start-page: 1 year: 2011 ident: ref24 publication-title: ?A random coordinate descent method on large optimization problems with linear constraints ? – ident: ref12 doi: 10.1109/TSP.2009.2016892 – ident: ref22 doi: 10.1137/100802001 – ident: ref36 doi: 10.1109/TAC.1978.1101718 – start-page: 28 year: 2012 ident: ref20 article-title: Feature clustering for accelerating parallel coordinate descent publication-title: Proc 26th Annu Conf Neural Inf Process Syst (NIPS) – ident: ref14 doi: 10.1137/120891009 – year: 2003 ident: ref35 publication-title: Finite-Dimensional Variational Inequalities and Complementarity Problem – year: 2012 ident: ref27 publication-title: ?Parallel coordinate descent methods for big data optimization ? – ident: ref38 doi: 10.1080/00207728108963798 – year: 1989 ident: ref33 publication-title: Parallel and Distributed Computation Numerical Methods – ident: ref13 doi: 10.1109/ACSSC.2013.6810364 – year: 2011 ident: ref41 publication-title: Neuro-Dynamic Programming
SSID	ssj0014496
Score	2.5527644
Snippet	We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable...
SourceID	crossref ieee
SourceType	Enrichment Source Index Database Publisher
StartPage	1874
SubjectTerms	Approximation methods Convergence distributed methods Jacobi method Jacobian matrices LASSO Optimization Parallel optimization Signal processing algorithms sparse solution variables selection Vectors
Title	Parallel Selective Algorithms for Nonconvex Big Data Optimization
URI	https://ieeexplore.ieee.org/document/7029716
Volume	63
WOSCitedRecordID	wos000350880900020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0476 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014496 issn: 1053-587X databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LS8MwGA_b8KAHX1OcL3LwItitbdImOc7H8CBzsCm7lTT9NgfbKl0n_vkmaVd2EMFbKfkg_JrH7-v3-CF0wyVl4CnhALjUoeAqR1LQ-8rX3Fhoj8Kz7YvfX1i_z8djMaihu6oWBgBs8hm0zaON5SepWptfZR1mlJa8sI7qjIVFrVYVMaDUanFpukCcgLPxJiTpis5oODA5XEHb1HFyI-6-dQVtaarYK6V38L_JHKL9kjribvGtj1ANlsdob6uhYBN1BzIz4ihzPLT6Nvoow935NM1m-cdihTVBxf10aTPNv_H9bIofZS7xqz42FmU95gl66z2NHp6dUiTBUXqr5o4kRHswfhjzmJmoIZG-kIkfKj9MPJpMhGABcDeWEpKAA48pC7XbQgiRns88IKeosUyXcIZwYNhYoggHzbJiKeIJITFAoCSEwldBC3U2uEWq7CBuhCzmkfUkXBFppCODdFQi3UK3lcVn0T3jj7FNA3I1rsT3_PfXF2jXGBc5NJeokWdruEI76iufrbJruzR-AHP-tWM
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JT8JAFH5BNFEPbmjEdQ5eTKy0nS4zR1wIRkQS0HBrptMHkgA1UIw_35mhEA7GxNukmTbNN9v35i0fwBUTXoiO5Bai7Vke2tISHqp15SpuzJVF4Zjyxe-NsNlk3S5vFeBmmQuDiCb4DG910_jyk1TO9FVZJdRKS06wButaOSvP1lr6DDzPqHEpwkAtn4XdhVPS5pVOu6WjuPxbncnJtLz7yiG0oqpiDpXa7v9-Zw92cvJIqvPR3ocCjg9ge6WkYAmqLTHR8ihD0jYKN2ozI9VhP50Mso_RlCiKSprp2MSaf5O7QZ88iEyQV7VxjPKMzEN4qz127utWLpNgSbVYM0tQqmwYN4hZHGq_IRUuF4kbSDdIHC_pcR76yOxYCEx8hiz2wkAZLpRS4bihg_QIiuN0jMdAfM3HEkkZKp4VCx73KI0RfSkw4K70y1BZ4BbJvIa4lrIYRsaWsHmkkI400lGOdBmul298zutn_NG3pEFe9svxPfn98SVs1jsvjajx1Hw-hS39oXlEzRkUs8kMz2FDfmWD6eTCTJMfFoq4rA
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Parallel+Selective+Algorithms+for+Nonconvex+Big+Data+Optimization&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Facchinei%2C+Francisco&rft.au=Scutari%2C+Gesualdo&rft.au=Sagratella%2C+Simone&rft.date=2015-04-01&rft.issn=1053-587X&rft.eissn=1941-0476&rft.volume=63&rft.issue=7&rft.spage=1874&rft.epage=1889&rft_id=info:doi/10.1109%2FTSP.2015.2399858&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TSP_2015_2399858
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon