Randomized Algorithms for Low-Rank Matrix Factorizations: Sharp Performance Bounds
The development of randomized algorithms for numerical linear algebra, e.g. for computing approximate QR and SVD factorizations, has recently become an intense area of research. This paper studies one of the most frequently discussed algorithms in the literature for dimensionality reduction—specific...
Saved in:
| Published in: | Algorithmica Vol. 72; no. 1; pp. 264 - 281 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Boston
Springer US
01.05.2015
|
| Subjects: | |
| ISSN: | 0178-4617, 1432-0541 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | The development of randomized algorithms for numerical linear algebra, e.g. for computing approximate QR and SVD factorizations, has recently become an intense area of research. This paper studies one of the most frequently discussed algorithms in the literature for dimensionality reduction—specifically for approximating an input matrix with a low-rank element. We introduce a novel and rather intuitive analysis of the algorithm in [
6
], which allows us to derive sharp estimates and give new insights about its performance. This analysis yields theoretical guarantees about the approximation error and at the same time, ultimate limits of performance (lower bounds) showing that our upper bounds are tight. Numerical experiments complement our study and show the tightness of our predictions compared with empirical observations. |
|---|---|
| AbstractList | The development of randomized algorithms for numerical linear algebra, e.g. for computing approximate QR and SVD factorizations, has recently become an intense area of research. This paper studies one of the most frequently discussed algorithms in the literature for dimensionality reduction—specifically for approximating an input matrix with a low-rank element. We introduce a novel and rather intuitive analysis of the algorithm in [
6
], which allows us to derive sharp estimates and give new insights about its performance. This analysis yields theoretical guarantees about the approximation error and at the same time, ultimate limits of performance (lower bounds) showing that our upper bounds are tight. Numerical experiments complement our study and show the tightness of our predictions compared with empirical observations. |
| Author | Witten, Rafi Candès, Emmanuel |
| Author_xml | – sequence: 1 givenname: Rafi surname: Witten fullname: Witten, Rafi organization: Bit Body, Inc – sequence: 2 givenname: Emmanuel surname: Candès fullname: Candès, Emmanuel email: candes@stanford.edu organization: Departments of Mathematics and of Statistics, Stanford University |
| BookMark | eNp9kEFOwzAQRS1UJNrCAdj5AgaP7cQJu1JRQAoCFVhbjuO0KY1d2amAnp6UsmLR1Sz-f6OZN0ID551F6BLoFVAqryOlIuGEgiB5lgORJ2gIgjNCEwEDNKQgMyJSkGdoFOOKUmAyT4doPteu8m2zsxWerBc-NN2yjbj2ARf-k_TpB37SXWi-8Eybrs93umu8izf4danDBr_Y0Jdb7YzFt37rqniOTmu9jvbib47R--zubfpAiuf7x-mkIIYz6IjlUuasKkud6JIBNYZZaTKapVroPONVmUvDa5PWiRA2ycoyBZ6wHKAqs4rVfIzkYa8JPsZga2Wa7ve4LuhmrYCqvRp1UKN6NWqvRsmehH_kJjStDt9HGXZgYt91CxvUym-D6x88Av0AVP95SQ |
| CitedBy_id | crossref_primary_10_1097_MNM_0000000000001618 crossref_primary_10_1016_j_ifacol_2015_10_308 crossref_primary_10_1088_2632_2153_ab8240 crossref_primary_10_1109_TIT_2022_3146206 crossref_primary_10_1016_j_patrec_2016_12_019 crossref_primary_10_1016_j_procs_2016_05_378 crossref_primary_10_1016_j_procs_2015_05_422 crossref_primary_10_1080_03081087_2016_1267104 crossref_primary_10_1016_j_procs_2016_05_379 crossref_primary_10_1137_17M1138480 crossref_primary_10_1038_s41592_018_0308_4 crossref_primary_10_1145_3242670 crossref_primary_10_1109_TKDE_2022_3180808 crossref_primary_10_1080_10618600_2017_1340302 crossref_primary_10_1111_insr_12176 crossref_primary_10_1137_16M1064556 crossref_primary_10_1145_3004053 crossref_primary_10_1109_TIT_2024_3450412 crossref_primary_10_1080_10618600_2022_2034636 crossref_primary_10_1016_j_advengsoft_2017_12_007 |
| Cites_doi | 10.1137/080736417 10.1016/j.acha.2007.12.002 10.1016/j.acha.2010.02.003 10.1214/aop/1176994775 10.1137/090771806 10.1017/CBO9780511810817 10.1137/040616413 10.1214/aop/1176992819 10.1142/9789814324359_0111 |
| ContentType | Journal Article |
| Copyright | Springer Science+Business Media New York 2014 |
| Copyright_xml | – notice: Springer Science+Business Media New York 2014 |
| DBID | AAYXX CITATION |
| DOI | 10.1007/s00453-014-9891-7 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1432-0541 |
| EndPage | 281 |
| ExternalDocumentID | 10_1007_s00453_014_9891_7 |
| GroupedDBID | -4Z -59 -5G -BR -EM -Y2 -~C -~X .86 .DC .VR 06D 0R~ 0VY 199 1N0 1SB 203 23M 28- 2J2 2JN 2JY 2KG 2KM 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 8TC 8UJ 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDPE ABDZT ABECU ABFSI ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABLJU ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTAH ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AI. AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BGNMA BSONS CAG COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP E.L EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ H~9 I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ KDC KOV KOW LAS LLZTM M4Y MA- N2Q N9A NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P9O PF- PT4 PT5 QOK QOS R4E R89 R9I RHV RIG RNI RNS ROL RPX RSV RZK S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TN5 TSG TSK TSV TUC U2A UG4 UOJIU UQL UTJUX UZXMN VC2 VFIZW VH1 VXZ W23 W48 WK8 YLTOR Z45 Z7X Z83 Z88 Z8R Z8W Z92 ZMTXR ZY4 ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION |
| ID | FETCH-LOGICAL-c321t-e37792dbba5ab210cc2e7c8086a4a983db97c3fc6f544e58bb61352911db8d2f3 |
| IEDL.DBID | RSV |
| ISICitedReferencesCount | 33 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000352434300012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0178-4617 |
| IngestDate | Sat Nov 29 02:20:27 EST 2025 Tue Nov 18 21:55:46 EST 2025 Fri Feb 21 02:43:11 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Matrix approximation Dimension reduction Random matrix Randomized linear algebra Pass efficient algorithm |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c321t-e37792dbba5ab210cc2e7c8086a4a983db97c3fc6f544e58bb61352911db8d2f3 |
| PageCount | 18 |
| ParticipantIDs | crossref_citationtrail_10_1007_s00453_014_9891_7 crossref_primary_10_1007_s00453_014_9891_7 springer_journals_10_1007_s00453_014_9891_7 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-05-01 |
| PublicationDateYYYYMMDD | 2015-05-01 |
| PublicationDate_xml | – month: 05 year: 2015 text: 2015-05-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Boston |
| PublicationPlace_xml | – name: Boston |
| PublicationTitle | Algorithmica |
| PublicationTitleAbbrev | Algorithmica |
| PublicationYear | 2015 |
| Publisher | Springer US |
| Publisher_xml | – name: Springer US |
| References | Rokhlin, Szlam, Tygert (CR7) 2009; 31 Silverstein (CR10) 1985; 13 Martinsson, Rokhlin, Tygert (CR6) 2011; 30 CR8 Halko, Martinsson, Tropp (CR3) 2011; 53 Geman (CR2) 1980; 8 CR9 Chen, Dongarrar (CR1) 2005; 27 Woolfe, Liberty, Rokhlin, Tygert (CR11) 2008; 25 Horn, Johnson (CR4) 1985 Kent, Mardia, Bibby (CR5) 1976 9891_CR9 F Woolfe (9891_CR11) 2008; 25 JW Silverstein (9891_CR10) 1985; 13 JT Kent (9891_CR5) 1976 9891_CR8 Z Chen (9891_CR1) 2005; 27 RA Horn (9891_CR4) 1985 V Rokhlin (9891_CR7) 2009; 31 N Halko (9891_CR3) 2011; 53 S Geman (9891_CR2) 1980; 8 P-G Martinsson (9891_CR6) 2011; 30 |
| References_xml | – volume: 31 start-page: 1100 year: 2009 end-page: 1124 ident: CR7 article-title: A randomized algorithm for principal component analysis publication-title: SIAM J. Matrix Anal. Appl. doi: 10.1137/080736417 – ident: CR9 – volume: 25 start-page: 335 year: 2008 end-page: 366 ident: CR11 article-title: A fast randomized algorithm for the approximation of matrices publication-title: Appl. Comput. Harmon. Anal. doi: 10.1016/j.acha.2007.12.002 – volume: 30 start-page: 47 year: 2011 end-page: 68 ident: CR6 article-title: A randomized algorithm for the decomposition of matrices publication-title: Appl. Comput. Harmon. Anal. doi: 10.1016/j.acha.2010.02.003 – ident: CR8 – volume: 8 start-page: 252 year: 1980 end-page: 261 ident: CR2 article-title: A limit theorem for the norm of random matrices publication-title: Ann. Probab. doi: 10.1214/aop/1176994775 – volume: 53 start-page: 217 year: 2011 end-page: 288 ident: CR3 article-title: Finding structure with randomness probabilistic algorithms for constructing approximate matrix decompositions publication-title: SIAM Rev. doi: 10.1137/090771806 – year: 1985 ident: CR4 publication-title: Matrix analysis doi: 10.1017/CBO9780511810817 – year: 1976 ident: CR5 publication-title: Multivariate analysis – volume: 27 start-page: 603 year: 2005 end-page: 620 ident: CR1 article-title: Condition numbers of gaussian random matrices publication-title: SIAM J. Matrix Anal. Appl. doi: 10.1137/040616413 – volume: 13 start-page: 1364 year: 1985 end-page: 1368 ident: CR10 article-title: The smallest eigenvalue of a large dimensional wishart matrix publication-title: Ann. Probab. doi: 10.1214/aop/1176992819 – volume-title: Multivariate analysis year: 1976 ident: 9891_CR5 – volume: 13 start-page: 1364 year: 1985 ident: 9891_CR10 publication-title: Ann. Probab. doi: 10.1214/aop/1176992819 – volume: 30 start-page: 47 year: 2011 ident: 9891_CR6 publication-title: Appl. Comput. Harmon. Anal. doi: 10.1016/j.acha.2010.02.003 – ident: 9891_CR9 – ident: 9891_CR8 doi: 10.1142/9789814324359_0111 – volume: 25 start-page: 335 year: 2008 ident: 9891_CR11 publication-title: Appl. Comput. Harmon. Anal. doi: 10.1016/j.acha.2007.12.002 – volume: 27 start-page: 603 year: 2005 ident: 9891_CR1 publication-title: SIAM J. Matrix Anal. Appl. doi: 10.1137/040616413 – volume: 8 start-page: 252 year: 1980 ident: 9891_CR2 publication-title: Ann. Probab. doi: 10.1214/aop/1176994775 – volume-title: Matrix analysis year: 1985 ident: 9891_CR4 doi: 10.1017/CBO9780511810817 – volume: 53 start-page: 217 year: 2011 ident: 9891_CR3 publication-title: SIAM Rev. doi: 10.1137/090771806 – volume: 31 start-page: 1100 year: 2009 ident: 9891_CR7 publication-title: SIAM J. Matrix Anal. Appl. doi: 10.1137/080736417 |
| SSID | ssj0012796 |
| Score | 2.3053522 |
| Snippet | The development of randomized algorithms for numerical linear algebra, e.g. for computing approximate QR and SVD factorizations, has recently become an intense... |
| SourceID | crossref springer |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 264 |
| SubjectTerms | Algorithm Analysis and Problem Complexity Algorithms Computer Science Computer Systems Organization and Communication Networks Data Structures and Information Theory Mathematics of Computing Theory of Computation |
| Title | Randomized Algorithms for Low-Rank Matrix Factorizations: Sharp Performance Bounds |
| URI | https://link.springer.com/article/10.1007/s00453-014-9891-7 |
| Volume | 72 |
| WOSCitedRecordID | wos000352434300012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1432-0541 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0012796 issn: 0178-4617 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEB60evBifWJ9sQdPykKTbLJZb1UsHmopVUtvIfuIlrZJaeIDf727aRIpqKCXXDIJYbI73ywz830AZ9LxHKNmhZkkNiZMX8Km9LAXarjiXERe3k046NBu1x8OWa-Y407LbveyJJlH6mrYzWQfpveHYOYzC9NVWNNo5xu9hv79oCod2DQX5TKy85hofC5Lmd-9YhmMliuhOcC06__6tC3YLPJJ1FosgG1YUfEO1EutBlRs3V3o98NYJtPRh5KoNXlK5qPseZoinbKiTvKG9d0xujN0_e-onUvwlPOZl8hwOs9Q72vCAF0ZLaZ0Dx7bNw_Xt7jQU8DCsa0MK0MuaEvOQzfk-qgnhK2o8PWhJiQh8x3JGRVOJLzIJUS5Puca611bh0PJfWlHzj7U4iRWB4AkkdKjTFBDRaoE41IoJizGLaXDK_ca0CwdG4iCbNxoXkyCiiY591mgfRYYnwW0AefVI7MF08ZvxhflnwiKTZf-bH34J-sj2NBZkbvoajyGWjZ_USewLl6zUTo_zRfbJ2VDzlc |
| linkProvider | Springer Nature |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEF60CnqxPrE-9-BJWWiSTTbrrYqlYlpKraW3kH1Ei21SmvjAX-9umkQKKugll0xCmOzON8vMfB8AZ8JyLK1mhajAJsJUXYK6cJATKLhijIdO1k048Ein4w6HtJvPcSdFt3tRkswidTnsprMP3fuDEXWpgcgyWMEKsDRhfu9-UJYOTJKJcmnZeYQVPhelzO9esQhGi5XQDGCa1X992ibYyPNJ2JgvgC2wJKNtUC20GmC-dXdArxdEIp6MPqSAjfFjPBulT5MEqpQVevEbUnefYVvT9b_DZibBU8xnXkLN6TyF3a8JA3iltZiSXfDQvOlft1Cup4C4ZRopkppc0BSMBXbA1FGPc1MS7qpDTYAD6lqCUcKtkDuhjbG0XcYU1tumCoeCucIMrT1QieJI7gMosBAOoZxoKlLJKRNcUm5QZkgVXplTA_XCsT7Pyca15sXYL2mSM5_5yme-9plPauC8fGQ6Z9r4zfii-BN-vumSn60P_mR9CtZa_bbne7edu0OwrjIke97heAQq6exFHoNV_pqOktlJtvA-ARss0Ts |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEF60inixPrE-9-BJWdokm2zWW30UxVpK1dJbyD6ixTYpTXzgr3c3LymoIF5yySSEye7ODPPN9wFwJCzH0mpWiApsIkzVxW8IBzm-CleM8cBJ0YT9Nul03MGAdnOd07hAuxctyWymQbM0hUl9IoJ6OfimMxGNA8KIutRAZB4sYI2j1-X6Xb9sI5gkFejSEvQIq1hdtDW_e8VsYJrtiqbBplX992eugpU8z4TNbGGsgTkZroNqoeEA8y29AXo9PxTRePghBWyOHqPpMHkax1ClsrAdvSF19xneahr_d9hKpXmKuc1TqLmeJ7D7NXkAz7RGU7wJHlqX9-dXKNdZQNwyjQRJTTpoCsZ822eqBOTclIS7qtjxsU9dSzBKuBVwJ7AxlrbLmMoBbFMdk4K5wgysLVAJo1BuAyiwEA6hnGiKUskpE1xSblBmSHXsMqcGGoWTPZ6TkGstjJFX0ienPvOUzzztM4_UwHH5yCRj4PjN-KT4K16-GeOfrXf-ZH0IlroXLa993bnZBcsqcbIz4OMeqCTTF7kPFvlrMoynB-ka_AThPdof |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Randomized+Algorithms+for+Low-Rank+Matrix+Factorizations%3A+Sharp+Performance+Bounds&rft.jtitle=Algorithmica&rft.au=Witten%2C+Rafi&rft.au=Cand%C3%A8s%2C+Emmanuel&rft.date=2015-05-01&rft.issn=0178-4617&rft.eissn=1432-0541&rft.volume=72&rft.issue=1&rft.spage=264&rft.epage=281&rft_id=info:doi/10.1007%2Fs00453-014-9891-7&rft.externalDBID=n%2Fa&rft.externalDocID=10_1007_s00453_014_9891_7 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0178-4617&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0178-4617&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0178-4617&client=summon |