Convergence of Stochastic Proximal Gradient Algorithm
We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under fu...
Gespeichert in:
| Veröffentlicht in: | Applied mathematics & optimization Jg. 82; H. 3; S. 891 - 917 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
Springer US
01.12.2020
Springer Nature B.V |
| Schlagworte: | |
| ISSN: | 0095-4616, 1432-0606 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms. |
|---|---|
| AbstractList | We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms. |
| Author | Villa, Silvia Vũ, Bằng Công Rosasco, Lorenzo |
| Author_xml | – sequence: 1 givenname: Lorenzo surname: Rosasco fullname: Rosasco, Lorenzo organization: DIBRIS, Università di Genova, LCSL, Istituto Italiano di Tecnologia and Massachusetts Institute of Technology – sequence: 2 givenname: Silvia surname: Villa fullname: Villa, Silvia email: silvia.villa@unige.it organization: Dipartimento di Matematica, Università di Genova – sequence: 3 givenname: Bằng Công surname: Vũ fullname: Vũ, Bằng Công organization: EPFL STI IEL LIONS, ELD 243 (Batiment EL) |
| BookMark | eNp9kE1LAzEURYNUsK3-AVcDrqMvM_mYLEvRKhQU1HVIM0k7ZZrUJBX9946OILjo6m7uefdxJmjkg7cIXRK4JgDiJgGUlGEgEoPkRGBxgsaEViUGDnyExgCSYcoJP0OTlLbQ9ytejRGbB_9u49p6Y4vgiucczEan3JriKYaPdqe7YhF101qfi1m3DrHNm905OnW6S_biN6fo9e72ZX6Pl4-Lh_lsiU1FZMYrICsLjXOUyKoPRkXtNHN1zQ0XpNSlo8zUjawFo64GoR0HSZnWDdGmktUUXQ139zG8HWzKahsO0feTqqSCcEZ4SfpWObRMDClF69Q-9o_HT0VAfetRgx7V61E_epToofofZNqscxt8jrrtjqPVgKZ-x69t_PvqCPUFpMp64Q |
| CitedBy_id | crossref_primary_10_1007_s10957_021_01821_2 crossref_primary_10_1109_TSP_2023_3237392 crossref_primary_10_1137_22M1480161 crossref_primary_10_1016_j_jmva_2024_105316 crossref_primary_10_1016_j_nonrwa_2023_103869 crossref_primary_10_1080_02331934_2023_2189015 crossref_primary_10_1093_bib_bbab256 crossref_primary_10_1137_20M1339842 crossref_primary_10_1214_21_STS835 crossref_primary_10_1007_s11590_021_01844_8 crossref_primary_10_1088_1361_6420_acc889 crossref_primary_10_1137_21M1402467 crossref_primary_10_1088_1361_6420_adb06f crossref_primary_10_1109_TIFS_2024_3405183 crossref_primary_10_1007_s00365_023_09635_3 crossref_primary_10_1007_s11432_023_4095_y crossref_primary_10_1080_02331934_2023_2230976 crossref_primary_10_1007_s10589_023_00504_0 crossref_primary_10_1007_s10107_022_01897_2 crossref_primary_10_1109_TCI_2020_3032101 crossref_primary_10_1137_20M1354398 crossref_primary_10_1080_10556788_2023_2256447 crossref_primary_10_1155_2023_4055950 crossref_primary_10_1109_TAC_2024_3381866 crossref_primary_10_1080_10556788_2021_1998492 crossref_primary_10_1007_s10915_022_02064_7 crossref_primary_10_1109_TSP_2021_3069677 crossref_primary_10_1007_s10589_022_00380_0 crossref_primary_10_1080_02331934_2025_2541709 crossref_primary_10_1109_TSP_2025_3554876 crossref_primary_10_1007_s10957_022_02122_y |
| Cites_doi | 10.1007/s10107-010-0434-y 10.1214/aoms/1177729586 10.1007/978-3-642-15883-4_27 10.7551/mitpress/8996.003.0006 10.1007/978-1-4419-9569-8_10 10.1007/s10957-016-0893-2 10.1214/12-STS394 10.1137/050626090 10.1137/070704277 10.1007/s10589-013-9633-9 10.1017/CBO9780511546921 10.1137/0330046 10.1016/j.jco.2009.01.002 10.1137/110848864 10.1287/10-SSY010 10.1007/s10107-010-0420-4 10.1137/S1052623497331063 10.1214/aoms/1177729392 10.1137/040615961 10.1137/140971233 10.1007/s10589-013-9628-6 10.1007/978-1-4899-2696-8 10.1007/s10589-017-9962-1 10.1007/BF01071091 10.1007/s10107-016-1030-6 10.1137/110844805 10.1080/01630563.2016.1254243 10.1017/CBO9781107298019 10.1007/978-3-642-75894-2 10.1007/978-3-642-82118-9 10.1111/j.1467-9868.2005.00503.x 10.1007/s10957-016-0978-y 10.1137/110848876 10.1137/16M1080173 10.1007/BF01069860 10.1007/978-1-4419-9467-7 10.1007/s10107-018-1297-x |
| ContentType | Journal Article |
| Copyright | Springer Science+Business Media, LLC, part of Springer Nature 2019 Springer Science+Business Media, LLC, part of Springer Nature 2019. |
| Copyright_xml | – notice: Springer Science+Business Media, LLC, part of Springer Nature 2019 – notice: Springer Science+Business Media, LLC, part of Springer Nature 2019. |
| DBID | AAYXX CITATION 3V. 7WY 7WZ 7XB 87Z 88I 8AO 8FE 8FG 8FK 8FL 8G5 ABJCF ABUWG AFKRA ARAPS AZQEC BENPR BEZIV BGLVJ CCPQU DWQXO FRNLG F~G GNUQQ GUQSH HCIFZ JQ2 K60 K6~ K7- L.- L6V M0C M2O M2P M7S MBDVC P5Z P62 PHGZM PHGZT PKEHL PQBIZ PQBZA PQEST PQGLB PQQKQ PQUKI PRINS PTHSS Q9U |
| DOI | 10.1007/s00245-019-09617-7 |
| DatabaseName | CrossRef ProQuest Central (Corporate) ABI/INFORM Collection ABI/INFORM Global (PDF only) ProQuest Central (purchase pre-March 2016) ABI/INFORM Collection Science Database (Alumni Edition) ProQuest Pharma Collection ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ABI/INFORM Collection (Alumni) Research Library (Alumni) Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials - QC ProQuest Central Business Premium Collection Technology collection ProQuest One Community College ProQuest Central Business Premium Collection (Alumni) ABI/INFORM Global (Corporate) ProQuest Central Student ProQuest Research Library SciTech Premium Collection ProQuest Computer Science Collection ProQuest Business Collection (Alumni Edition) ProQuest Business Collection Computer Science Database ABI/INFORM Professional Advanced ProQuest Engineering Collection ABI/INFORM Global Research Library Science Database Engineering Database Research Library (Corporate) Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection Proquest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Business (UW System Shared) ProQuest One Business (Alumni) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection ProQuest Central Basic |
| DatabaseTitle | CrossRef ABI/INFORM Global (Corporate) ProQuest Business Collection (Alumni Edition) ProQuest One Business Research Library Prep Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College Research Library (Alumni Edition) ProQuest Pharma Collection ProQuest Central China ABI/INFORM Complete ProQuest Central ABI/INFORM Professional Advanced ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest Central Korea ProQuest Research Library ProQuest Central (New) ABI/INFORM Complete (Alumni Edition) Engineering Collection Advanced Technologies & Aerospace Collection Business Premium Collection ABI/INFORM Global Engineering Database ProQuest Science Journals (Alumni Edition) ABI/INFORM Global (Alumni Edition) ProQuest Central Basic ProQuest Science Journals ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection ProQuest Business Collection Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition Materials Science & Engineering Collection ProQuest One Business (Alumni) ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) Business Premium Collection (Alumni) |
| DatabaseTitleList | ABI/INFORM Global (Corporate) |
| Database_xml | – sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Mathematics |
| EISSN | 1432-0606 |
| EndPage | 917 |
| ExternalDocumentID | 10_1007_s00245_019_09617_7 |
| GroupedDBID | -52 -5D -5G -BR -EM -Y2 -~C -~X .86 .VR 06D 0R~ 0VY 1N0 1SB 2.D 203 23M 28- 2J2 2JN 2JY 2KG 2KM 2LR 2P1 2VQ 2~H 30V 3V. 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 7WY 88I 8AO 8FE 8FG 8FL 8G5 8TC 8UJ 8VB 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDBF ABDPE ABDZT ABECU ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTAH ABTEG ABTHY ABTKH ABTMW ABULA ABUWG ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACGOD ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACNCT ACOKC ACOMO ACPIV ACUHS ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMOZ AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFFNX AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHQJS AHSBF AHYZX AI. AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ AKVCP ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN AZQEC B-. B0M BA0 BAPOH BBWZM BDATZ BENPR BEZIV BGLVJ BGNMA BPHCQ BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DWQXO EAD EAP EBA EBLON EBR EBS EBU EIOEI EJD EMI EMK EPL ESBYG EST ESX FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRNLG FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNUQQ GNWQR GQ6 GQ7 GROUPED_ABI_INFORM_COMPLETE GUQSH GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I-F I09 IHE IJ- IKXTQ ITM IWAJR IXC IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ K1G K60 K6V K6~ K7- KDC KOV KOW L6V LAS LLZTM M0C M2O M2P M4Y M7S MA- N2Q N9A NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P2P P62 P9R PF0 PQBIZ PQBZA PQQKQ PROAC PT4 PT5 PTHSS Q2X QOK QOS QWB R4E R89 R9I RHV RIG RNI RNS ROL RPX RSV RZK S16 S1Z S26 S27 S28 S3B SAP SCLPG SDD SDH SDM SHX SISQX SJYHP SMT SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 T9H TH9 TN5 TSG TSK TSV TUC TUS TWZ U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW VH1 W23 W48 WH7 WHG WJK WK8 YLTOR Z45 Z81 Z8U ZL0 ZMTXR ZWQNP ZY4 ~8M ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG ADXHL AEZWR AFDZB AFFHD AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP AMVHM ATHPR AYFIA CITATION PHGZM PHGZT PQGLB 7XB 8FK JQ2 L.- MBDVC PKEHL PQEST PQUKI PRINS Q9U |
| ID | FETCH-LOGICAL-c319t-b01be0dff4193dff5478fa5f886c6712a2f45c8d98754f807af60945aad1ac393 |
| IEDL.DBID | RSV |
| ISICitedReferencesCount | 51 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000490200600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0095-4616 |
| IngestDate | Wed Nov 05 14:51:46 EST 2025 Sat Nov 29 02:47:08 EST 2025 Tue Nov 18 22:23:46 EST 2025 Fri Feb 21 02:26:27 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | Forward–backward splitting algorithm Stochastic optimization Proximal methods |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c319t-b01be0dff4193dff5478fa5f886c6712a2f45c8d98754f807af60945aad1ac393 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 2471651621 |
| PQPubID | 47316 |
| PageCount | 27 |
| ParticipantIDs | proquest_journals_2471651621 crossref_primary_10_1007_s00245_019_09617_7 crossref_citationtrail_10_1007_s00245_019_09617_7 springer_journals_10_1007_s00245_019_09617_7 |
| PublicationCentury | 2000 |
| PublicationDate | 20201200 2020-12-00 20201201 |
| PublicationDateYYYYMMDD | 2020-12-01 |
| PublicationDate_xml | – month: 12 year: 2020 text: 20201200 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | Applied mathematics & optimization |
| PublicationTitleAbbrev | Appl Math Optim |
| PublicationYear | 2020 |
| Publisher | Springer US Springer Nature B.V |
| Publisher_xml | – name: Springer US – name: Springer Nature B.V |
| References | Atchade, Fort, Moulines (CR1) 2017; 18 Ermol’ev, Tuniev (CR20) 1968; 2 Bottou, Curtis, Nocedal (CR10) 2018; 60 Kwok, Hu, Pan (CR28) 2009; 22 Zou, Hastie (CR52) 2005; 67 Nemirovski, Yudin (CR35) 1983 CR38 Shalev-Shwartz, Singer, Srebro, Cotter (CR45) 2011; 127 Bauschke, Combettes (CR4) 2011 Benveniste, Métivier, Priouret (CR5) 1990 CR32 Bach, Jenatton, Mairal, Obozinski (CR2) 2012; 27 Lan (CR29) 2012; 133 Rosasco, Villa, Vu (CR41) 2017; 38 CR3 CR6 Kushner, Yin (CR27) 1997 Ermol’ev (CR19) 1969; 5 Rosasco, Villa, Vu (CR40) 2016; 169 CR47 CR46 Lin, Chen, Peña (CR31) 2014; 58 Robbins, Monro (CR39) 1951; 22 Shor (CR48) 1985 Cesa-Bianchi, Lugosi (CR11) 2006 Combettes, Wajs (CR15) 2005; 4 Juditski, Nesterov (CR25) 2014; 4 Ghadimi, Lan (CR22) 2013; 23 Villa, Salzo, Baldassarre, Verri (CR50) 2013; 23 CR14 Polyak, Juditsky (CR37) 1992; 30 Lin, Rosasco, Villa, Zhou (CR30) 2018; 71 Defazio, Bach, Lacoste-Julien (CR17) 2014; 27 Combettes, Pesquet (CR12) 2015; 25 Hazan, Kale (CR23) 2014; 15 Nemirovski, Juditsky, Lan, Shapiro (CR34) 2008; 19 Kiefer, Wolfowitz (CR26) 1952; 23 Shalev-Shwartz, Ben-David (CR44) 2014 Schmidt, Le Roux, Bach (CR43) 2011; 24 Bertsekas, Tsitsiklis (CR7) 2000; 10 Blatt, Hero, Gauchman (CR9) 2007; 18 De Mol, De Vito, Rosasco (CR16) 2009; 25 Combettes, Pesquet (CR13) 2016; 1 CR24 Xiao (CR51) 2010; 11 Bianchi, Hachem (CR8) 2016; 171 Villa, Rosasco, Mosci, Verri (CR49) 2014; 58 Polyak (CR36) 1987 Nekrylova (CR33) 1975; 11 Schmidt, Le Roux, Bach (CR42) 2017; 162 Ghadimi, Lan (CR21) 2012; 22 Duchi, Singer (CR18) 2009; 10 M Schmidt (9617_CR42) 2017; 162 A Juditski (9617_CR25) 2014; 4 MW Schmidt (9617_CR43) 2011; 24 N Cesa-Bianchi (9617_CR11) 2006 B Polyak (9617_CR36) 1987 9617_CR14 Q Lin (9617_CR31) 2014; 58 A Nemirovski (9617_CR34) 2008; 19 S Shalev-Shwartz (9617_CR44) 2014 A Nemirovski (9617_CR35) 1983 PL Combettes (9617_CR15) 2005; 4 J Kiefer (9617_CR26) 1952; 23 J Duchi (9617_CR18) 2009; 10 9617_CR24 D Blatt (9617_CR9) 2007; 18 H Robbins (9617_CR39) 1951; 22 S Villa (9617_CR50) 2013; 23 L Bottou (9617_CR10) 2018; 60 Z Zou (9617_CR52) 2005; 67 L Rosasco (9617_CR40) 2016; 169 A Benveniste (9617_CR5) 1990 S Shalev-Shwartz (9617_CR45) 2011; 127 C De Mol (9617_CR16) 2009; 25 9617_CR32 E Hazan (9617_CR23) 2014; 15 PL Combettes (9617_CR12) 2015; 25 S Ghadimi (9617_CR21) 2012; 22 9617_CR38 NZ Shor (9617_CR48) 1985 YuM Ermol’ev (9617_CR19) 1969; 5 HH Bauschke (9617_CR4) 2011 G Lan (9617_CR29) 2012; 133 ZV Nekrylova (9617_CR33) 1975; 11 YF Atchade (9617_CR1) 2017; 18 F Bach (9617_CR2) 2012; 27 L Rosasco (9617_CR41) 2017; 38 HJ Kushner (9617_CR27) 1997 A Defazio (9617_CR17) 2014; 27 BT Polyak (9617_CR37) 1992; 30 S Villa (9617_CR49) 2014; 58 9617_CR3 9617_CR47 9617_CR6 DP Bertsekas (9617_CR7) 2000; 10 P Bianchi (9617_CR8) 2016; 171 9617_CR46 L Xiao (9617_CR51) 2010; 11 J Lin (9617_CR30) 2018; 71 PL Combettes (9617_CR13) 2016; 1 S Ghadimi (9617_CR22) 2013; 23 JT Kwok (9617_CR28) 2009; 22 YuM Ermol’ev (9617_CR20) 1968; 2 |
| References_xml | – volume: 27 start-page: 1646 year: 2014 end-page: 1654 ident: CR17 article-title: SAGA: a fast incremental gradient method with support for non-strongly convex composite objectives publication-title: Adv. Neural Inf. Process. Syst. – volume: 58 start-page: 455 year: 2014 end-page: 482 ident: CR31 article-title: A sparsity preserving stochastic gradient methods for sparse regression publication-title: Comput. Optim. Appl. – volume: 22 start-page: 400 year: 1951 end-page: 407 ident: CR39 article-title: A stochastic approximation method publication-title: Ann. Math. Stat. – volume: 67 start-page: 301 year: 2005 end-page: 320 ident: CR52 article-title: Regularization and variable selection via the elastic net publication-title: J. R. Stat. Soc. Ser. B – volume: 60 start-page: 223 year: 2018 end-page: 311 ident: CR10 article-title: Optimization methods for large-scale machine learning publication-title: SIAM Rev. – year: 1983 ident: CR35 publication-title: Problem Complexity and Method Efficiency in Optimization – volume: 24 start-page: 1458 year: 2011 end-page: 1466 ident: CR43 article-title: Convergence rates of inexact proximal-gradient methods for convex optimization publication-title: Adv. Neural Inf. Process. Syst. – ident: CR46 – volume: 4 start-page: 44 year: 2014 end-page: 80 ident: CR25 article-title: Deterministic and stochastic primal-dual subgradient methods for uniformly convex minimization publication-title: Stoch. Syst. – year: 2006 ident: CR11 publication-title: Prediction, Learning, and Games – volume: 18 start-page: 29 year: 2007 end-page: 51 ident: CR9 article-title: A convergent incremental gradient method with a constant step size publication-title: SIAM J. Optim. – volume: 30 start-page: 838 year: 1992 end-page: 855 ident: CR37 article-title: Acceleration of stochastic approximation by averaging publication-title: SIAM J. Control Optim. – volume: 10 start-page: 627 year: 2000 end-page: 642 ident: CR7 article-title: Gradient convergence in gradient methods with errors publication-title: SIAM J. Optim. – volume: 23 start-page: 2061 year: 2013 end-page: 2089 ident: CR22 article-title: Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, II: shrinking procedures and optimal algorithms publication-title: SIAM J. Optim. – volume: 11 start-page: 218 year: 1975 end-page: 222 ident: CR33 article-title: The rate of convergence of the stochastic gradient method publication-title: Cybernetics – volume: 38 start-page: 602 year: 2017 end-page: 626 ident: CR41 article-title: A first-order stochastic primal-dual algorithm with correction step publication-title: Numer. Funct. Anal. Optim. – ident: CR32 – volume: 1 start-page: 13 year: 2016 end-page: 37 ident: CR13 article-title: Stochastic approximations and perturbations in forward-backward splitting for monotone operators publication-title: Pure Appl. Funct. Anal. – volume: 2 start-page: 76 year: 1968 end-page: 83 ident: CR20 article-title: Random fejér and quasi-fejér sequences publication-title: Theory Optim. Solut. Akademiya Nauk Ukrainskoĭ SSR Kiev – volume: 25 start-page: 201 year: 2009 end-page: 230 ident: CR16 article-title: Elastic-net regularization in learning theory publication-title: J. Complex. – year: 1997 ident: CR27 publication-title: Stochastic Approximation Algorithms and Applications – volume: 169 start-page: 388 year: 2016 end-page: 406 ident: CR40 article-title: Stochastic forward-backward splitting for monotone inclusions publication-title: J. Optim. Theory Appl. – volume: 133 start-page: 365 year: 2012 end-page: 397 ident: CR29 article-title: An optimal method for stochastic composite optimization publication-title: Math. Progr. – volume: 10 start-page: 2899 year: 2009 end-page: 2934 ident: CR18 article-title: Efficient online and batch learning using forward backward splitting publication-title: J. Mach. Learn. Res. – volume: 15 start-page: 2489 year: 2014 end-page: 2512 ident: CR23 article-title: Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization publication-title: J. Mach. Learn. Res. – volume: 71 start-page: 95 year: 2018 end-page: 113 ident: CR30 article-title: Modified Fejér sequences and applications publication-title: Comput. Optim. Appl. – volume: 25 start-page: 1221 year: 2015 end-page: 1248 ident: CR12 article-title: Stochastic quasi-fejér block-coordinate fixed point iterations with random sweeping publication-title: SIAM J. Optim. – ident: CR47 – volume: 4 start-page: 1168 year: 2005 end-page: 1200 ident: CR15 article-title: Signal recovery by proximal forward-backward splitting publication-title: Multiscale Model. Simul. – ident: CR14 – volume: 18 start-page: 310 year: 2017 end-page: 342 ident: CR1 article-title: On perturbed proximal gradient algorithms publication-title: J. Mach. Learn. Res. – year: 1987 ident: CR36 publication-title: Introduction to Optimization – ident: CR6 – volume: 162 start-page: 83 year: 2017 end-page: 112 ident: CR42 article-title: Minimizing finite sums with the stochastic average gradient publication-title: Math. Progr. Ser. B – volume: 11 start-page: 2543 year: 2010 end-page: 2596 ident: CR51 article-title: Dual averaging methods for regularized stochastic learning and online optimization publication-title: J. Mach. Learn. Res. – year: 1985 ident: CR48 publication-title: Minimization Methods for Non-differentiable Functions – year: 2011 ident: CR4 publication-title: Convex Analysis and Monotone Operator Theory in Hilbert Spaces – year: 1990 ident: CR5 publication-title: Adaptive Algorithms and Stochastic Approximations – volume: 22 start-page: 781 year: 2009 end-page: 789 ident: CR28 article-title: Accelerated gradient methods for stochastic optimization and online learning publication-title: Adv. Neural Inf. Process. Syst. – volume: 5 start-page: 208 year: 1969 end-page: 220 ident: CR19 article-title: On the method of generalized stochastic gradients and quasi-fejér sequences publication-title: Cybernetics – volume: 58 start-page: 381 year: 2014 end-page: 407 ident: CR49 article-title: Proximal methods for the latent group lasso penalty publication-title: Comput. Optim. Appl. – volume: 23 start-page: 1607 year: 2013 end-page: 1633 ident: CR50 article-title: Accelerated and inexact forward-backward algorithms publication-title: SIAM J. Optim. – ident: CR3 – ident: CR38 – year: 2014 ident: CR44 publication-title: Understanding Machine Learning: From Theory to Algorithms – volume: 19 start-page: 1574 year: 2008 end-page: 1609 ident: CR34 article-title: Robust stochastic approximation approach to stochastic programming publication-title: SIAM J. Optim. – volume: 171 start-page: 90 year: 2016 end-page: 120 ident: CR8 article-title: Dynamical behavior of a stochastic forward-backward algorithm using random monotone operators publication-title: J. Optim. Theory Appl. – volume: 127 start-page: 3 year: 2011 end-page: 30 ident: CR45 article-title: Pegasos: primal estimated sub-gradient solver for SVM publication-title: Math. Progr. Ser. B – volume: 27 start-page: 450 year: 2012 end-page: 468 ident: CR2 article-title: Structured sparsity through convex optimization publication-title: Stat. Sci. – volume: 22 start-page: 1469 year: 2012 end-page: 1492 ident: CR21 article-title: Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, I: a generic algorithmic framework publication-title: SIAM J. Optim. – ident: CR24 – volume: 23 start-page: 462 year: 1952 end-page: 466 ident: CR26 article-title: Stochastic estimation of the maximum of a regression function publication-title: Ann. Math. Stat. – ident: 9617_CR47 – volume: 133 start-page: 365 year: 2012 ident: 9617_CR29 publication-title: Math. Progr. doi: 10.1007/s10107-010-0434-y – volume: 22 start-page: 400 year: 1951 ident: 9617_CR39 publication-title: Ann. Math. Stat. doi: 10.1214/aoms/1177729586 – volume: 27 start-page: 1646 year: 2014 ident: 9617_CR17 publication-title: Adv. Neural Inf. Process. Syst. – ident: 9617_CR32 doi: 10.1007/978-3-642-15883-4_27 – ident: 9617_CR6 doi: 10.7551/mitpress/8996.003.0006 – ident: 9617_CR14 doi: 10.1007/978-1-4419-9569-8_10 – volume: 169 start-page: 388 year: 2016 ident: 9617_CR40 publication-title: J. Optim. Theory Appl. doi: 10.1007/s10957-016-0893-2 – volume: 10 start-page: 2899 year: 2009 ident: 9617_CR18 publication-title: J. Mach. Learn. Res. – volume: 27 start-page: 450 year: 2012 ident: 9617_CR2 publication-title: Stat. Sci. doi: 10.1214/12-STS394 – volume: 4 start-page: 1168 year: 2005 ident: 9617_CR15 publication-title: Multiscale Model. Simul. doi: 10.1137/050626090 – volume: 15 start-page: 2489 year: 2014 ident: 9617_CR23 publication-title: J. Mach. Learn. Res. – ident: 9617_CR38 – volume: 19 start-page: 1574 year: 2008 ident: 9617_CR34 publication-title: SIAM J. Optim. doi: 10.1137/070704277 – volume: 24 start-page: 1458 year: 2011 ident: 9617_CR43 publication-title: Adv. Neural Inf. Process. Syst. – volume: 58 start-page: 455 year: 2014 ident: 9617_CR31 publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-013-9633-9 – volume-title: Prediction, Learning, and Games year: 2006 ident: 9617_CR11 doi: 10.1017/CBO9780511546921 – volume: 30 start-page: 838 year: 1992 ident: 9617_CR37 publication-title: SIAM J. Control Optim. doi: 10.1137/0330046 – volume: 2 start-page: 76 year: 1968 ident: 9617_CR20 publication-title: Theory Optim. Solut. Akademiya Nauk Ukrainskoĭ SSR Kiev – volume: 25 start-page: 201 year: 2009 ident: 9617_CR16 publication-title: J. Complex. doi: 10.1016/j.jco.2009.01.002 – ident: 9617_CR46 – volume: 22 start-page: 1469 year: 2012 ident: 9617_CR21 publication-title: SIAM J. Optim. doi: 10.1137/110848864 – volume: 4 start-page: 44 year: 2014 ident: 9617_CR25 publication-title: Stoch. Syst. doi: 10.1287/10-SSY010 – volume: 127 start-page: 3 year: 2011 ident: 9617_CR45 publication-title: Math. Progr. Ser. B doi: 10.1007/s10107-010-0420-4 – volume: 10 start-page: 627 year: 2000 ident: 9617_CR7 publication-title: SIAM J. Optim. doi: 10.1137/S1052623497331063 – volume: 23 start-page: 462 year: 1952 ident: 9617_CR26 publication-title: Ann. Math. Stat. doi: 10.1214/aoms/1177729392 – volume: 18 start-page: 29 year: 2007 ident: 9617_CR9 publication-title: SIAM J. Optim. doi: 10.1137/040615961 – volume: 1 start-page: 13 year: 2016 ident: 9617_CR13 publication-title: Pure Appl. Funct. Anal. – volume: 25 start-page: 1221 year: 2015 ident: 9617_CR12 publication-title: SIAM J. Optim. doi: 10.1137/140971233 – volume: 58 start-page: 381 year: 2014 ident: 9617_CR49 publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-013-9628-6 – ident: 9617_CR3 – volume-title: Problem Complexity and Method Efficiency in Optimization year: 1983 ident: 9617_CR35 – volume-title: Stochastic Approximation Algorithms and Applications year: 1997 ident: 9617_CR27 doi: 10.1007/978-1-4899-2696-8 – volume: 71 start-page: 95 year: 2018 ident: 9617_CR30 publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-017-9962-1 – volume: 5 start-page: 208 year: 1969 ident: 9617_CR19 publication-title: Cybernetics doi: 10.1007/BF01071091 – volume: 18 start-page: 310 year: 2017 ident: 9617_CR1 publication-title: J. Mach. Learn. Res. – volume: 162 start-page: 83 year: 2017 ident: 9617_CR42 publication-title: Math. Progr. Ser. B doi: 10.1007/s10107-016-1030-6 – volume: 23 start-page: 1607 year: 2013 ident: 9617_CR50 publication-title: SIAM J. Optim. doi: 10.1137/110844805 – volume: 11 start-page: 2543 year: 2010 ident: 9617_CR51 publication-title: J. Mach. Learn. Res. – volume: 38 start-page: 602 year: 2017 ident: 9617_CR41 publication-title: Numer. Funct. Anal. Optim. doi: 10.1080/01630563.2016.1254243 – volume-title: Understanding Machine Learning: From Theory to Algorithms year: 2014 ident: 9617_CR44 doi: 10.1017/CBO9781107298019 – volume-title: Introduction to Optimization year: 1987 ident: 9617_CR36 – volume-title: Adaptive Algorithms and Stochastic Approximations year: 1990 ident: 9617_CR5 doi: 10.1007/978-3-642-75894-2 – volume-title: Minimization Methods for Non-differentiable Functions year: 1985 ident: 9617_CR48 doi: 10.1007/978-3-642-82118-9 – volume: 67 start-page: 301 year: 2005 ident: 9617_CR52 publication-title: J. R. Stat. Soc. Ser. B doi: 10.1111/j.1467-9868.2005.00503.x – volume: 171 start-page: 90 year: 2016 ident: 9617_CR8 publication-title: J. Optim. Theory Appl. doi: 10.1007/s10957-016-0978-y – volume: 23 start-page: 2061 year: 2013 ident: 9617_CR22 publication-title: SIAM J. Optim. doi: 10.1137/110848876 – volume: 60 start-page: 223 year: 2018 ident: 9617_CR10 publication-title: SIAM Rev. doi: 10.1137/16M1080173 – volume: 22 start-page: 781 year: 2009 ident: 9617_CR28 publication-title: Adv. Neural Inf. Process. Syst. – volume: 11 start-page: 218 year: 1975 ident: 9617_CR33 publication-title: Cybernetics doi: 10.1007/BF01069860 – volume-title: Convex Analysis and Monotone Operator Theory in Hilbert Spaces year: 2011 ident: 9617_CR4 doi: 10.1007/978-1-4419-9467-7 – ident: 9617_CR24 doi: 10.1007/s10107-018-1297-x |
| SSID | ssj0002363 |
| Score | 2.5577977 |
| Snippet | We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish... |
| SourceID | proquest crossref springer |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 891 |
| SubjectTerms | Algorithms Calculus of Variations and Optimal Control; Optimization Control Convergence Convexity Machine learning Mathematical and Computational Physics Mathematical Methods in Physics Mathematics Mathematics and Statistics Numerical and Computational Physics Simulation Systems Theory Theoretical |
| SummonAdditionalLinks | – databaseName: Computer Science Database dbid: K7- link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEA6-DnrwLa6u0oM3DW7SJE1Psiw-QBRBhb2VdJrowrrV3Sr-fJOYdlXQi6dS2qal32Qemck3CB0AQGwtDcPayA5mOZVYppBjK8-pUKbogFK-2URyfS37_fQmLLhNQlllrRO9oi5KcGvkx9RqUcGJoOTk-QW7rlEuuxpaaMyieUIpcXJ-meBGE9M4dFJLOWaCiLBpxm-d8zlHG0i7giFrxXHy3TBNvc0fCVJvd85W_vvFq2g5eJxR91NE1tCMHq2jpS88hPbsqiFvnWwg3nOV6H5Tpo5KE91WJTwqx-cc3YzL98GTHe187EvFqqg7fLDvrB6fNtH92eld7wKH7goY7LSrcN4hue4UxjDrw9mDI_YyihspBYiEUEUN4yCL1EY0zAKZKCNsLMiVKoiCOI230NyoHOltFMWMa9ApANcFozRPJbCcUykK62_EpGghUv_aDAL1uOuAMcwa0mQPR2bhyDwcWdJCh80zz5_EG3_e3a4xyMIknGRTAFroqEZxevn30Xb-Hm0XLVIXdfuiljaaq8aveg8twFs1mIz3vQh-AOSV3yk priority: 102 providerName: ProQuest |
| Title | Convergence of Stochastic Proximal Gradient Algorithm |
| URI | https://link.springer.com/article/10.1007/s00245-019-09617-7 https://www.proquest.com/docview/2471651621 |
| Volume | 82 |
| WOSCitedRecordID | wos000490200600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1432-0606 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002363 issn: 0095-4616 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3dT9wwDLcG7AEe2AcgjrFTH_Y2IrVpkqaPcIJNmrhV3D6AlypNE-4kuKJeQfz5OKEtG9omsRdXUdOocuzYVuyfAT5orWO0NIwYK0PCCiqJTHVBUJ5ToWwZaqV8s4lkPJanp2nWFoUtumz37krSn9R9sZu_JcTQ16X4oN0lyRKsoLmTTh1PJj_685fGbf-0lBMmItGWyvx5jd_N0aOP-eRa1Fubo1f_95-vYb31LoP9B3F4Ay_M_C2s_YI5iKPjHqh1sQF85LLOfQGmCSobTJpKT5XDbg6yurqbXeFqn2qfFtYE-5cXVT1rpleb8P3o8NvoM2k7KRCNKtaQIowKE5bWMvTX8OFAvKziVkqhRRJRRS3jWpYpRi8MNy1RVmDcx5UqI6XjNN6C5Xk1N9sQxIwbbVKtuSkZpUUqNSs4laJE3yKOygFEHUNz3cKMu24Xl3kPkOwZlCODcs-gPBnAx_6b6weQjX_O3u32KW8VbpFTNLKCR4JGA9jr9uXx9d9X23ne9HewSl3E7RNadmG5qW_Me3ipb5vZoh7CUvLzbAgrB4fj7ARHXxKC9DgcOUq_epo5mkyQZvx86EX3HhId3cU |
| linkProvider | Springer Nature |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LT9wwEB4hitRy6LtigbY5tKfWauLYjnNAFdqWgoAVUqnELXUmdkGCDeyG15_iN3bsTXZLpXLj0FMUJbHkfOPxjGfmG4B3iJjSTiOYdTpmouSa6RxLRvKcK-OqGI0JzSaywUAfHOR7c3DT1cL4tMpOJwZFXdXoz8g_cdKiSiaKJ59Pz5jvGuWjq10LjYlYbNvrS3LZxmtbXwjf95xvfN3vb7K2qwBDEreGlXFS2rhyTpDtQhdPaOWMdForVFnCDXdCoq7IGZeCJpAZp8gHksZUicHUky-Ryn8gBC0HnyoY96ean6dt57ZcMqES1RbphFK9EOMkx90nKJHVwLLbG-HMuv0rIBv2uY0n_9sfegqPW4s6Wp8sgWcwZ4fPYfEPnkW6252S045fgOz7TPtQdGqj2kXfmxoPjeerjvZG9dXRCY32bRRS4Zpo_fgXzbE5PHkJP-5lFq9gflgP7RJEqZAWbY4obSU4L3ONopRcq4rsqTSpepB0UBbYUqv7Dh_HxZQUOsBfEPxFgL_IevBh-s3phFjkzrdXO8yLVsmMixngPfjYSc3s8b9HW757tLfwcHN_d6fY2Rpsr8Aj7k8YQgLPKsw3o3P7Ghbwojkaj94E8Y_g531L02-D4TzL |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3PT9RAFH4haIgeBEXCKmIPepIJ2-nMdHowhoArBN1sgiaES5m-zgAJbHG3Cv5r_nW8mW131URuHDg1TdtJpu-b92Pmve8BvEHEhCyNYNbpLhMF10xnWDDCc6aMK7toTGg2kfb7-vAwG8zB77YWxqdVtjoxKOqyQr9HvslJiyoZKx5vuiYtYrDT-3D5nfkOUv6ktW2nMYHIvv11ReHb-P3eDsn6Lee9j1-3d1nTYYAhQa9mRTcubLd0TpAfQxdPbuWMdForVGnMDXdCoi4pMJeCJpMapygeksaUscHEEzGR-n-QUozpA7-BPJpaAZ40XdwyyYSKVVOwE8r2wnknBfE-WYk8CJb-bRRnnu4_h7PB5vUW7_PfWoInjacdbU2WxlOYs8Nn8PgP_kW6-zIlrR0vg9z2GfihGNVGlYsO6gpPjeexjgaj6vrsgkb7NAopcnW0dX5Cc6xPL57DtzuZxQrMD6uhXYUoEdKizRClLQXnRaZRFJJrVZKflcRlB-JWrDk2lOu-88d5PiWLDlDICQp5gEKeduDd9JvLCeHIrW-vtfLPG-UzzmfC78BGi6DZ4_-P9uL20V7DAoEo_7zX338Jj7jfeAh5PWswX49-2FfwEH_WZ-PRelgJERzfNZhuAH8WRdw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Convergence+of+Stochastic+Proximal+Gradient+Algorithm&rft.jtitle=Applied+mathematics+%26+optimization&rft.au=Rosasco%2C+Lorenzo&rft.au=Villa%2C+Silvia&rft.au=V%C5%A9%2C+B%E1%BA%B1ng+C%C3%B4ng&rft.date=2020-12-01&rft.pub=Springer+US&rft.issn=0095-4616&rft.eissn=1432-0606&rft.volume=82&rft.issue=3&rft.spage=891&rft.epage=917&rft_id=info:doi/10.1007%2Fs00245-019-09617-7&rft.externalDocID=10_1007_s00245_019_09617_7 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0095-4616&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0095-4616&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0095-4616&client=summon |