Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifolds
First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR...
Uložené v:
| Vydané v: | IEEE transactions on pattern analysis and machine intelligence Ročník 43; číslo 2; s. 459 - 472 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
United States
IEEE
01.02.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Predmet: | |
| ISSN: | 0162-8828, 1939-3539, 2160-9292, 1939-3539 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with n components, R-SPIDER is proved to converge to an 6-approximate stationary point within O(min(n/ε 2 ,1/3)) stochastic gradient evaluations, beating the best-known complexity O(n+1/ ε 4 ); for online optimization, R-SPIDER is shown to converge with O(1/ε 3 ) complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods. |
|---|---|
| AbstractList | First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with n components, R-SPIDER is proved to converge to an ϵ-approximate stationary point within [Formula: see text] stochastic gradient evaluations, beating the best-known complexity [Formula: see text]; for online optimization, R-SPIDER is shown to converge with [Formula: see text] complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods.First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with n components, R-SPIDER is proved to converge to an ϵ-approximate stationary point within [Formula: see text] stochastic gradient evaluations, beating the best-known complexity [Formula: see text]; for online optimization, R-SPIDER is shown to converge with [Formula: see text] complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods. First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with n components, R-SPIDER is proved to converge to an ϵ-approximate stationary point within [Formula: see text] stochastic gradient evaluations, beating the best-known complexity [Formula: see text]; for online optimization, R-SPIDER is shown to converge with [Formula: see text] complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods. First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with n components, R-SPIDER is proved to converge to an 6-approximate stationary point within O(min(n/ε 2 ,1/3)) stochastic gradient evaluations, beating the best-known complexity O(n+1/ ε 4 ); for online optimization, R-SPIDER is shown to converge with O(1/ε 3 ) complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods. First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component analysis and low-rank matrix completion. The current paper presents an efficient Riemannian Stochastic Path Integrated Differential EstimatoR (R-SPIDER) algorithm to solve the finite-sum and online Riemannian non-convex minimization problems. At the core of R-SPIDER is a recursive semi-stochastic gradient estimator that can accurately estimate Riemannian gradient under not only exponential mapping and parallel transport, but also general retraction and vector transport operations. Compared with prior Riemannian algorithms, such a recursive gradient estimation mechanism endows R-SPIDER with lower computational cost in first-order oracle complexity. Specifically, for finite-sum problems with [Formula Omitted] components, R-SPIDER is proved to converge to an [Formula Omitted]-approximate stationary point within [Formula Omitted] stochastic gradient evaluations, beating the best-known complexity [Formula Omitted]; for online optimization, R-SPIDER is shown to converge with [Formula Omitted] complexity which is, to the best of our knowledge, the first non-asymptotic result for online Riemannian optimization. For the special case of gradient dominated functions, we further develop a variant of R-SPIDER with improved linear rate of convergence. Extensive experimental results demonstrate the advantage of the proposed algorithms over the state-of-the-art Riemannian non-convex optimization methods. |
| Author | Yan, Shuicheng Feng, Jiashi Zhou, Pan Yuan, Xiao-Tong |
| Author_xml | – sequence: 1 givenname: Pan orcidid: 0000-0003-3400-8943 surname: Zhou fullname: Zhou, Pan email: pzhou@u.nus.edu organization: Department of Electrical and Computer Engineering, National University of Singapore, Singapore – sequence: 2 givenname: Xiao-Tong orcidid: 0000-0002-7151-8806 surname: Yuan fullname: Yuan, Xiao-Tong email: xtyuan1980@gmail.com organization: School of Automation, Nanjing University of Information Science & Technology, Nanjing, China – sequence: 3 givenname: Shuicheng surname: Yan fullname: Yan, Shuicheng email: eleyans@nus.edu.sg organization: Department of Electrical and Computer Engineering, National University of Singapore, Singapore – sequence: 4 givenname: Jiashi orcidid: 0000-0001-6843-0064 surname: Feng fullname: Feng, Jiashi email: elefjia@nus.edu.sg organization: Department of Electrical and Computer Engineering, National University of Singapore, Singapore |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/31398110$$D View this record in MEDLINE/PubMed |
| BookMark | eNp9kU1LXDEUhoMoOlr_gIVyoZtu7jTJuR_JUoZOKzid0tpFVyE3Hxi5N5kmmVL99Y3O6MJFIXDC4XlPwnlO0aEP3iB0QfCcEMw_3ny7XF3NKSZ8TjkAa8gBmlHS4ZpTTg_RDJOO1oxRdoJOU7rDmDQthmN0AgQ4KzNm6NdSpmxitXQx5XoddbmvTL4NOlU2xOpHDuq2IE5VX4OvF8H_MX-r9Sa7yT3I7IKvyvnuzCS9d9JXK-mdDaNOb9CRlWMy5_t6hn4uP90svtTX689Xi8vrWgFvc82k5Za0hhKjWwnQNcCI0haGzmIrJahGD1aZDg-91j0BqlVrpVVQOnLo4Ax92M3dxPB7a1IWk0vKjKP0JmyToLQnrAXW0oK-f4XehW305XeCNn1PG8YxKdS7PbUdJqPFJrpJxnvxvLQC0B2gYkgpGvuCECwezYgnM-LRjNibKSH2KqRcflpgjtKN_4--3UWdMeblLdbz4hrgH-zKnAE |
| CODEN | ITPIDJ |
| CitedBy_id | crossref_primary_10_1109_TCOMM_2023_3277872 crossref_primary_10_1137_21M1405551 crossref_primary_10_1287_moor_2022_1302 crossref_primary_10_1016_j_sigpro_2025_109904 crossref_primary_10_1109_TNNLS_2021_3056947 crossref_primary_10_1007_s10589_023_00477_0 crossref_primary_10_1007_s10712_021_09644_6 crossref_primary_10_1109_TPAMI_2023_3234160 crossref_primary_10_1016_j_eswa_2022_118921 crossref_primary_10_1109_TCYB_2021_3049845 crossref_primary_10_1109_TPAMI_2019_2933841 crossref_primary_10_1016_j_jfranklin_2024_107311 crossref_primary_10_1109_TPAMI_2021_3112139 crossref_primary_10_3390_su151813815 |
| Cites_doi | 10.1109/CDC.2014.7039534 10.1016/S0893-6080(00)00026-5 10.1007/978-94-015-8390-9 10.1137/100802529 10.1109/TAC.2013.2254619 10.1016/0041-5553(63)90382-3 10.1016/j.neucom.2017.07.041 10.1007/s10107-012-0584-1 10.1109/TNNLS.2016.2601307 10.1109/34.927464 10.1016/0169-7439(87)80084-9 10.1137/110845768 10.1007/s10107-006-0706-8 10.1109/TPAMI.2019.2933841 10.1145/1970392.1970395 10.1007/s10107-014-0765-1 10.1016/S0893-6080(05)80089-9 10.1109/TIT.2016.2632149 10.1007/978-3-319-46128-1_50 10.1093/comjnl/4.3.265 10.1007/s11263-015-0816-y 10.1109/CVPR.2017.419 10.1137/140955483 10.1093/imanum/22.3.359 10.1109/TSP.2005.850378 10.1109/TIP.2017.2762595 10.1109/TPAMI.2003.1251154 10.1109/TIP.2016.2623487 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| DBID | 97E RIA RIE AAYXX CITATION NPM 7SC 7SP 8FD JQ2 L7M L~C L~D 7X8 |
| DOI | 10.1109/TPAMI.2019.2933841 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef PubMed Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional MEDLINE - Academic |
| DatabaseTitle | CrossRef PubMed Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed Technology Research Database |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 2160-9292 1939-3539 |
| EndPage | 472 |
| ExternalDocumentID | 31398110 10_1109_TPAMI_2019_2933841 8792163 |
| Genre | orig-research Journal Article |
| GrantInformation_xml | – fundername: NUS grantid: IDS R-263-000-C67-646; ECRA R-263-000-C87-133 – fundername: National Natural Science Foundation of China; Natural Science Foundation of China grantid: 61876090 funderid: 10.13039/501100001809 – fundername: MOE grantid: Tier-II R-263-000-D17-112 |
| GroupedDBID | --- -DZ -~X .DC 0R~ 29I 4.4 53G 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 UHB ~02 AAYXX CITATION 5VS 9M8 AAYOK ABFSI ADRHT AETIX AGSQL AI. AIBXA ALLEH FA8 H~9 IBMZZ ICLAB IFJZH NPM PKN RIC RIG RNI RZB VH1 XJT Z5M 7SC 7SP 8FD JQ2 L7M L~C L~D 7X8 |
| ID | FETCH-LOGICAL-c395t-8af9f15e21ed5a3364381cdf3b6f0faa3c4dbfce60b7dd7132dc5fafc360bab63 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 16 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000607383300007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0162-8828 1939-3539 |
| IngestDate | Sun Sep 28 06:08:52 EDT 2025 Sun Jun 29 15:55:41 EDT 2025 Wed Feb 19 02:29:58 EST 2025 Sat Nov 29 05:15:59 EST 2025 Tue Nov 18 22:11:38 EST 2025 Wed Aug 27 02:28:32 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c395t-8af9f15e21ed5a3364381cdf3b6f0faa3c4dbfce60b7dd7132dc5fafc360bab63 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23 |
| ORCID | 0000-0003-3400-8943 0000-0001-6843-0064 0000-0002-7151-8806 |
| OpenAccessLink | https://ink.library.smu.edu.sg/sis_research/9004 |
| PMID | 31398110 |
| PQID | 2477248901 |
| PQPubID | 85458 |
| PageCount | 14 |
| ParticipantIDs | ieee_primary_8792163 proquest_miscellaneous_2271853852 proquest_journals_2477248901 crossref_primary_10_1109_TPAMI_2019_2933841 crossref_citationtrail_10_1109_TPAMI_2019_2933841 pubmed_primary_31398110 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-02-01 |
| PublicationDateYYYYMMDD | 2021-02-01 |
| PublicationDate_xml | – month: 02 year: 2021 text: 2021-02-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States – name: New York |
| PublicationTitle | IEEE transactions on pattern analysis and machine intelligence |
| PublicationTitleAbbrev | TPAMI |
| PublicationTitleAlternate | IEEE Trans Pattern Anal Mach Intell |
| PublicationYear | 2021 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref56 ref12 ref15 ref14 ref52 ref11 ref54 ref10 zhou (ref49) 2018 meyer (ref5) 2011 ref17 nguyen (ref27) 2017 martinez (ref53) 1998 ref51 de sa (ref40) 2018 ref46 ref45 shen (ref50) 2017; 18 liu (ref35) 2017 ref42 ref41 ref44 ref43 zhang (ref20) 2016 kasai (ref8) 2016 zhou (ref48) 2018 ref9 ref4 ref6 kasai (ref23) 2016 hosseini (ref7) 2015 fang (ref28) 2018 ref37 ref31 ref30 zhou (ref47) 2018 ref32 ref2 ref1 zhang (ref18) 2016 kasai (ref22) 2018 john (ref29) 1961; 4 nesterov (ref34) 2006 park (ref39) 2018 kasai (ref24) 2018 courbariaux (ref38) 2015 ref21 nguyen (ref26) 2018 tan (ref3) 2014 absil (ref33) 2009 (ref55) 2014 zhang (ref19) 2018 johnson (ref25) 2013 zhang (ref36) 2018 da cruz neto (ref16) 1998; 3 |
| References_xml | – ident: ref6 doi: 10.1109/CDC.2014.7039534 – ident: ref10 doi: 10.1016/S0893-6080(00)00026-5 – ident: ref32 doi: 10.1007/978-94-015-8390-9 – ident: ref42 doi: 10.1137/100802529 – year: 2009 ident: ref33 publication-title: Optimization Algorithms on Matrix Manifolds – ident: ref21 doi: 10.1109/TAC.2013.2254619 – ident: ref45 doi: 10.1016/0041-5553(63)90382-3 – volume: 18 start-page: 7650 year: 2017 ident: ref50 article-title: A tight bound of hard thresholding publication-title: J Mach Learn Res – ident: ref11 doi: 10.1016/j.neucom.2017.07.041 – start-page: 1703 year: 2018 ident: ref19 article-title: An estimate sequence for geodesically convex optimization publication-title: Proc Conf Learn Theory – ident: ref43 doi: 10.1007/s10107-012-0584-1 – year: 2006 ident: ref34 publication-title: Introductory Lectures on Convex Optimization A Basic Course – ident: ref1 doi: 10.1109/TNNLS.2016.2601307 – ident: ref52 doi: 10.1109/34.927464 – start-page: 1242 year: 2018 ident: ref48 article-title: New insight into hybrid stochastic gradient descent: Beyond with-replacement sampling and convexity publication-title: Proc Conf Neutral Inf Process Syst – year: 2018 ident: ref40 article-title: High-accuracy low-precision training publication-title: arXiv 1803 03383 – ident: ref9 doi: 10.1016/0169-7439(87)80084-9 – start-page: 315 year: 2013 ident: ref25 article-title: Accelerating stochastic gradient descent using predictive variance reduction publication-title: Proc Conf Neutral Inf Process Syst – ident: ref4 doi: 10.1137/110845768 – start-page: 1617 year: 2016 ident: ref20 article-title: First-order methods for geodesically convex optimization publication-title: Proc Conf Learn Theory – ident: ref46 doi: 10.1007/s10107-006-0706-8 – year: 2015 ident: ref38 article-title: Training deep neural networks with low precision multiplications publication-title: Proc Workshop Int Conf Learn Represent – start-page: 910 year: 2015 ident: ref7 article-title: Matrix manifold optimization for Gaussian mixtures publication-title: Proc Conf Neutral Inf Process Syst – ident: ref31 doi: 10.1109/TPAMI.2019.2933841 – start-page: 4868 year: 2017 ident: ref35 article-title: Accelerated first-order methods for geodesically convex optimization on Riemannian manifolds publication-title: Proc Conf Neutral Inf Process Syst – ident: ref56 doi: 10.1145/1970392.1970395 – start-page: 3925 year: 2018 ident: ref49 article-title: Stochastic nested variance reduction for nonconvex optimization publication-title: Adv in Neural Info Proc Syst – start-page: 2521 year: 2018 ident: ref22 article-title: Riemannian stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis publication-title: Proc Int Conf Mach Learn – start-page: 545 year: 2011 ident: ref5 article-title: Linear regression under fixed-rank constraints: A Riemannian approach publication-title: Proc Int Conf Mach Learn – year: 2018 ident: ref39 article-title: Training deep neural network in limited precision publication-title: arXiv 1810 05486 – ident: ref30 doi: 10.1007/s10107-014-0765-1 – start-page: 1012 year: 2016 ident: ref8 article-title: Low-rank tensor completion: A Riemannian manifold preconditioning approach publication-title: Proc Int Conf Mach Learn – start-page: 1539 year: 2014 ident: ref3 article-title: Riemannian pursuit for big matrix recovery publication-title: Proc Int Conf Mach Learn – ident: ref15 doi: 10.1016/S0893-6080(05)80089-9 – year: 2018 ident: ref36 article-title: R-SPIDER: A fast Riemannian stochastic optimization algorithm with curvature independent rate publication-title: arXiv 1811 04194 – start-page: 2613 year: 2018 ident: ref26 article-title: SARAH: A novel method for machine learning problems using stochastic recursive gradient publication-title: Proc Int Conf Mach Learn – ident: ref2 doi: 10.1109/TIT.2016.2632149 – start-page: 4592 year: 2016 ident: ref18 article-title: Riemannian SVRG: Fast stochastic optimization on Riemannian manifolds publication-title: Proc Conf Neutral Inf Process Syst – start-page: 1988 year: 2018 ident: ref47 article-title: Efficient stochastic gradient hard thresholding publication-title: Proc Conf Neutral Inf Process Syst – start-page: 269 year: 2018 ident: ref24 article-title: Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis publication-title: Proc Int Conf Artif Intell Statist – ident: ref51 doi: 10.1007/978-3-319-46128-1_50 – volume: 3 start-page: 89 year: 1998 ident: ref16 article-title: Geodesic algorithms in Riemannian geometry publication-title: Balkan J Geom Appl – year: 2017 ident: ref27 article-title: Stochastic recursive gradient algorithm for nonconvex optimization publication-title: arXiv 1705 07261 – volume: 4 start-page: 265 year: 1961 ident: ref29 article-title: The QR transformation a unitary analogue to the lr transformationpart 1 publication-title: Comput J doi: 10.1093/comjnl/4.3.265 – ident: ref37 doi: 10.1007/s11263-015-0816-y – start-page: 689 year: 2018 ident: ref28 article-title: SPIDER: Near-optimal non-convex optimization via stochastic path-integrated differential estimator publication-title: Adv in Neural Info Proc Syst – year: 2016 ident: ref23 article-title: Riemannian stochastic variance reduced gradient on Grassmann manifold publication-title: arXiv 1605 07367 – year: 2014 ident: ref55 article-title: Optimization and estimation on manifolds – ident: ref13 doi: 10.1109/CVPR.2017.419 – ident: ref44 doi: 10.1137/140955483 – ident: ref41 doi: 10.1093/imanum/22.3.359 – ident: ref17 doi: 10.1109/TSP.2005.850378 – year: 1998 ident: ref53 article-title: The AR face database – ident: ref14 doi: 10.1109/TIP.2017.2762595 – ident: ref54 doi: 10.1109/TPAMI.2003.1251154 – ident: ref12 doi: 10.1109/TIP.2016.2623487 |
| SSID | ssj0014503 |
| Score | 2.4532654 |
| Snippet | First-order non-convex Riemannian optimization algorithms have gained recent popularity in structured machine learning problems including principal component... |
| SourceID | proquest pubmed crossref ieee |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 459 |
| SubjectTerms | Algorithms Complexity Complexity theory Computational geometry Convergence Convex analysis Convexity Machine learning Manifolds Minimization non-convex optimization online learning Optimization Principal components analysis Riemann manifold Riemannian optimization Signal processing algorithms Stochastic processes stochastic variance-reduced algorithm |
| Title | Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifolds |
| URI | https://ieeexplore.ieee.org/document/8792163 https://www.ncbi.nlm.nih.gov/pubmed/31398110 https://www.proquest.com/docview/2477248901 https://www.proquest.com/docview/2271853852 |
| Volume | 43 |
| WOSCitedRecordID | wos000607383300007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2160-9292 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014503 issn: 0162-8828 databaseCode: RIE dateStart: 19790101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Rb9MwED5tEw_wsMEGrLBNRuINvKV2EtuP00QFEu0mGFJ5ipyzLSptCVpbxM_n7CbRHgBpUhRFjpNYuXPuu_juO4C3uqB21DV3Kqt5XmPJLdaCozalI_tkERNl_mc1m-n53FxtwfshF8Z7n4LP_Gk8TGv5rsV1_FV2ppURhB-2YVsptcnVGlYM8iJVQSYEQzOc3Ig-QSYzZ9dX59NPMYrLnJJxkzqP5WEkQR89jomz9-xRKrDyb6yZbM5k72GjfQq7HbZk5xtleAZbvtmHvb5uA-um8T48uUdCeADfJzaSJbDJgoAgv4xUnGya6kovGSFa9nXV4g8b6ZzZrG34RQxT_80u6VNz2-VwMtq-LPxtLH9kGza1zSK0N275HL5NPlxffORdwQWO0hQrrm0wYVx4MfausFKWkf8LXZB1GbJgrcTc1QF9mdXKOXJvhcMi2ICSWmxdyhew07SNPwQWlCFPztMuwzz4UgsnUBfoJbnBWvkRjPvXXmHHRh6LYtxUySvJTJWkVkWpVZ3URvBuuObnhovjv70PokyGnp04RnDUS7fqpuuyEjk5GTmNla56M5ymiRZXT2zj2zX1ESpiG12IEbzcaMVw716ZXv39ma_hsYihMCnY-wh2VndrfwyP8Ndqsbw7IW2e65OkzX8A8Vrv1g |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Rb9MwED6NgQQ8MNgGFAYYiTfwlthO4jxOE9Um2m6CIo2nyDnbotKWoLVF_HzObhLtAZCQoihynMTKnXPfxXffAbzTGbWjrrktkpqrGnNusBYcdZlbsk8GMVLmT4rZTF9elhdb8GHIhXHOxeAzdxgO41q-bXEdfpUd6aIUhB_uwN1MKZFusrWGNQOVxTrIhGFojpMj0afIJOXR_OJ4ehbiuMpDMm9Sq1AgRhL40WlInb1lkWKJlb-jzWh1xjv_N97H8KhDl-x4ow5PYMs1u7DTV25g3UTehYe3aAj34NvYBLoENl4QFOTngYyTTWNl6SUjTMu-rFr8bgKhM5u1DT8Jgeq_2Dl9bK67LE5G2-eFuw4FkEzDpqZZ-PbKLvfh6_jj_OSUdyUXOMoyW3FtfOnTzInU2cxImQcGMLRe1rlPvDESla09ujypC2vJwRUWM288SmoxdS6fwnbTNu45MF-U5Ms52iWovMu1sAJ1hk6SI6wLN4K0f-0VdnzkoSzGVRX9kqSsotSqILWqk9oI3g_X_Niwcfyz916QydCzE8cIDnrpVt2EXVZCkZuhaKx01dvhNE21sH5iGteuqY8oArrRmRjBs41WDPfulenFn5_5Bu6fzqeTanI2-_QSHogQGBNDvw9ge3Wzdq_gHv5cLZY3r6NO_wbooPI1 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Faster+First-Order+Methods+for+Stochastic+Non-Convex+Optimization+on+Riemannian+Manifolds&rft.jtitle=IEEE+transactions+on+pattern+analysis+and+machine+intelligence&rft.au=Zhou%2C+Pan&rft.au=Xiao-Tong%2C+Yuan&rft.au=Shuicheng+Yan&rft.au=Feng%2C+Jiashi&rft.date=2021-02-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0162-8828&rft.eissn=1939-3539&rft.volume=43&rft.issue=2&rft.spage=459&rft_id=info:doi/10.1109%2FTPAMI.2019.2933841&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0162-8828&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0162-8828&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0162-8828&client=summon |