Generalization bounds for sparse random feature expansions
Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However,...
Uložené v:
| Vydané v: | Applied and computational harmonic analysis Ročník 62; s. 310 - 330 |
|---|---|
| Hlavní autori: | , , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Elsevier Inc
01.01.2023
|
| Predmet: | |
| ISSN: | 1063-5203, 1096-603X |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications. We introduce the sparse random feature expansion to obtain parsimonious random feature models. We leverage ideas from compressive sensing to generate random feature expansions with theoretical guarantees even in the data-scarce setting. We provide generalization bounds for functions in a certain class depending on the number of samples and the distribution of features. By introducing sparse features, i.e. features with random sparse weights, we provide improved bounds for low order functions. We show that our method outperforms shallow networks in several scientific machine learning tasks. |
|---|---|
| AbstractList | Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications. We introduce the sparse random feature expansion to obtain parsimonious random feature models. We leverage ideas from compressive sensing to generate random feature expansions with theoretical guarantees even in the data-scarce setting. We provide generalization bounds for functions in a certain class depending on the number of samples and the distribution of features. By introducing sparse features, i.e. features with random sparse weights, we provide improved bounds for low order functions. We show that our method outperforms shallow networks in several scientific machine learning tasks. |
| Author | Schaeffer, Hayden Hashemi, Abolfazl Shi, Robert Topcu, Ufuk Tran, Giang Ward, Rachel |
| Author_xml | – sequence: 1 givenname: Abolfazl surname: Hashemi fullname: Hashemi, Abolfazl organization: Purdue University, United States of America – sequence: 2 givenname: Hayden surname: Schaeffer fullname: Schaeffer, Hayden email: hschaeff@andrew.cmu.edu organization: Carnegie Mellon University, United States of America – sequence: 3 givenname: Robert orcidid: 0000-0001-7850-8291 surname: Shi fullname: Shi, Robert organization: The University of Texas at Austin, United States of America – sequence: 4 givenname: Ufuk surname: Topcu fullname: Topcu, Ufuk organization: The University of Texas at Austin, United States of America – sequence: 5 givenname: Giang surname: Tran fullname: Tran, Giang organization: University of Waterloo, Canada – sequence: 6 givenname: Rachel surname: Ward fullname: Ward, Rachel organization: The University of Texas at Austin, United States of America |
| BookMark | eNp9kE1LAzEQhoNUsK3-AU_7B3adJPvRiBcpWoWCFwVvIR8TTGmTkmxF_fXuqicPPc0c5nl555mRSYgBCbmkUFGg7dWmUuZNVQwYq2BRAfATMqUg2rIF_joZ95aXDQN-RmY5bwAorRsxJdcrDJjU1n-p3sdQ6HgINhcupiLvVcpYJBVs3BUOVX9IWODHXoU8nOZzcurUNuPF35yTl_u75-VDuX5aPS5v16Xhdd2XtquRaaGFclwY7TiIRjPbsHpoiaztWmucFQBN54BrY00HTW07o7miqm34nCx-c02KOSd00vj-p22flN9KCnJ0IDdydCBHBxIWckgfUPYP3Se_U-nzOHTzC-Hw1LvHJLPxGAxan9D00kZ_DP8GpgN4eQ |
| CitedBy_id | crossref_primary_10_1111_mice_13372 crossref_primary_10_1088_1361_6501_acea9e crossref_primary_10_1007_s43670_023_00063_9 crossref_primary_10_1137_23M1624749 crossref_primary_10_1177_14759217251360635 crossref_primary_10_1615_Int_J_UncertaintyQuantification_2025051106 crossref_primary_10_1016_j_apacoust_2025_110562 crossref_primary_10_1214_25_AOS2492 crossref_primary_10_3390_atmos14091451 crossref_primary_10_1137_24M1648703 crossref_primary_10_1016_j_acha_2025_101789 crossref_primary_10_1093_sysbio_syae026 crossref_primary_10_1007_s10444_024_10147_2 crossref_primary_10_1109_TAP_2024_3501410 crossref_primary_10_1016_j_aei_2024_102394 crossref_primary_10_1016_j_ymssp_2023_111032 crossref_primary_10_1007_s43670_025_00100_9 crossref_primary_10_1109_TAC_2024_3486655 crossref_primary_10_1016_j_sigpro_2025_110239 crossref_primary_10_1016_j_ecolind_2024_112835 crossref_primary_10_1016_j_laa_2024_04_023 crossref_primary_10_1007_s11538_023_01174_z |
| Cites_doi | 10.1016/j.acha.2015.02.003 10.1109/JSTSP.2007.910281 10.1137/130916138 10.1109/TIT.2006.871582 10.1016/j.cma.2017.07.038 10.1016/j.jcp.2015.09.001 10.1111/j.1467-9868.2009.00718.x 10.1109/18.256500 10.1111/j.1541-0420.2012.01751.x 10.1007/s10208-017-9350-3 10.1109/TIT.2006.885507 10.1016/j.jat.2012.01.008 10.1007/BF01397280 10.1007/s00365-010-9105-8 10.1137/080714488 10.1109/TIT.2009.2021377 10.1016/j.jcp.2011.01.002 10.1103/RevModPhys.34.123 10.1002/cpa.20124 10.1137/18M1194730 10.1090/S0025-5718-09-02319-9 10.1137/18M116798X 10.1007/s10208-012-9115-y 10.1016/j.jcss.2004.04.001 10.1080/10618600.2014.938812 10.1214/aos/1176348546 |
| ContentType | Journal Article |
| Copyright | 2022 The Author(s) |
| Copyright_xml | – notice: 2022 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION |
| DOI | 10.1016/j.acha.2022.08.003 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Mathematics |
| EISSN | 1096-603X |
| EndPage | 330 |
| ExternalDocumentID | 10_1016_j_acha_2022_08_003 S1063520322000653 |
| GrantInformation_xml | – fundername: MURI grantid: FA9550-19-1-0005 funderid: https://doi.org/10.13039/100014036 – fundername: NSERC grantid: 2018-06135 funderid: https://doi.org/10.13039/501100000038 – fundername: NSF grantid: DMS-1752116; DMS-1952735 funderid: https://doi.org/10.13039/100000001 – fundername: MURI grantid: FA9550-21-1-0084 funderid: https://doi.org/10.13039/100014036 |
| GroupedDBID | --K --M .~1 0R~ 1B1 1RT 1~. 1~5 23M 4.4 457 4G. 5GY 5VS 6I. 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAFTH AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AASFE AAXUO ABAOU ABFNM ABJNI ABMAC ABVKL ABXDB ABYKQ ACAZW ACDAQ ACGFS ACRLP ADBBV ADEZE ADFGL ADMUD AEBSH AEKER AENEX AEXQZ AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AIEXJ AIGVJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ ARUGR ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC CAG COF CS3 DM4 EBS EFBJH EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA HVGLF HZ~ IHE IXB J1W KOM LG5 M26 M41 MCRUF MHUIS MO0 N9A NCXOZ O-L O9- OAUVE OK1 OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SSW SSZ T5K WUQ XPP ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO ADVLN AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD |
| ID | FETCH-LOGICAL-c344t-d74e2b9b9af39cbf3095b2d524003e2676dcfd90057f03bcdc7054d7cb3a1a653 |
| ISICitedReferencesCount | 31 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000870987200003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1063-5203 |
| IngestDate | Sat Nov 29 07:10:32 EST 2025 Tue Nov 18 21:55:19 EST 2025 Fri Feb 23 02:39:36 EST 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Sparse optimization Compressive sensing Random features 60B20 Generalization error 65D15 68Q32 46N10 |
| Language | English |
| License | This is an open access article under the CC BY license. |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c344t-d74e2b9b9af39cbf3095b2d524003e2676dcfd90057f03bcdc7054d7cb3a1a653 |
| ORCID | 0000-0001-7850-8291 |
| OpenAccessLink | https://dx.doi.org/10.1016/j.acha.2022.08.003 |
| PageCount | 21 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_acha_2022_08_003 crossref_primary_10_1016_j_acha_2022_08_003 elsevier_sciencedirect_doi_10_1016_j_acha_2022_08_003 |
| PublicationCentury | 2000 |
| PublicationDate | January 2023 2023-01-00 |
| PublicationDateYYYYMMDD | 2023-01-01 |
| PublicationDate_xml | – month: 01 year: 2023 text: January 2023 |
| PublicationDecade | 2020 |
| PublicationTitle | Applied and computational harmonic analysis |
| PublicationYear | 2023 |
| Publisher | Elsevier Inc |
| Publisher_xml | – name: Elsevier Inc |
| References | Schaeffer, Tran, Ward, Zhang (br0470) 2020; 18 Adcock, Brugiapaglia, Webster (br0010) 2017 Rosset, Swirszcz, Srebro, Zhu (br0570) 2007 Mei, Montanari (br0350) 2020 Guo, Zhu, Chow, Ibrahim (br0550) 2012 Kandasamy, Yu (br0610) 2016 Candes, Tao (br0090) 2006; 52 van Den Berg, Friedlander (br0680) 2009 Potts, Schmischke (br0390) 2019 Constantine, Emory, Larsson, Iaccarino (br0160) 2015; 302 Rauhut, Ward (br0430) 2012; 164 Arora, Du, Hu, Li, Wang (br0030) 2019 Potts, Schmischke (br0380) 2019 Barron (br0040) 1993; 39 Szabó, Sriperumbudur (br0670) 2019 D. Dua, C. Graff, UCI machine learning repository, 2017. Rahimi, Recht (br0410) 2008 Ullah, Mianjy, Marinov, Arora (br0660) 2018 Du, Zhai, Poczos, Singh (br0200) 2018 Figueiredo, Nowak, Wright (br0600) 2007 Chkifa, Dexter, Tran, Webster (br0130) 2018; 311 Moosmann, Triggs, Jurie (br0360) 2006 Maass, Markram (br0330) 2004; 69 Cai, Xu, Zhang (br0070) 2009; 55 DeVore, Petrova, Wojtaszczyk (br0170) 2011; 33 Shi, Lee, Wahba (br0540) 2007 Jones (br0280) 1992; 20 Du, Hu, Kakade, Lee, Lei (br0590) 2020 Rauhut, Ward (br0440) 2016; 40 Donoho (br0180) 2006; 52 Rakotomamonjy, Flamary, Yger (br0580) 2013 Marelli, Sudret (br0340) 2014 Jacot, Gabriel, Hongler (br0270) 2018 Sriperumbudur, Szabó (br0490) 2015 Chen, Wang, Deng, Huang (br0630) 2017 Rudi, Rosasco (br0650) 2017 Ravikumar, Lafferty, Liu, Wasserman (br0640) 2009 Chitta, Jin, Jain (br0120) 2012 Candes, Romberg, Tao (br0080) 2006; 59 Yang, Li, Mahdavi, Jin, Zhou (br0520) 2012; 25 Sutherland, Schneider (br0500) 2015 Schaeffer, Tran, Ward (br0460) 2018; 78 Constantine, Eftekhari, Hokanson, Ward (br0150) 2017; 326 Heisenberg (br0250) 1927 Constantine, Dow, Wang (br0140) 2014; 36 Saltelli, Ratto, Andres, Campolongo, Cariboni, Gatelli, Saisana, Tarantola (br0450) 2008 Li, Ton, Oglic, Sejdinovic (br0320) 2019 Lim, Hastie (br0560) 2015 Harris (br0240) 2019 Rahimi, Recht (br0420) 2007 Fornasier, Schnass, Vybiral (br0220) 2012; 12 Rahimi, Recht (br0400) 2008 Yen, Lin, Lin, Ravikumar, Dhillon (br0530) 2014 Liu, Chen, Huang (br0620) 2020 Block (br0060) 1962; 34 Foucart, Rauhut (br0230) 2013 Kuo, Sloan, Wasilkowski, Woźniakowski (br0290) 2010; 79 Adcock (br0020) 2018; 18 Doostan, Owhadi (br0190) 2011; 230 Li, Liang (br0310) 2018 Foucart (10.1016/j.acha.2022.08.003_br0230) 2013 Chitta (10.1016/j.acha.2022.08.003_br0120) 2012 Fornasier (10.1016/j.acha.2022.08.003_br0220) 2012; 12 Ravikumar (10.1016/j.acha.2022.08.003_br0640) 2009 DeVore (10.1016/j.acha.2022.08.003_br0170) 2011; 33 Chen (10.1016/j.acha.2022.08.003_br0630) 2017 Adcock (10.1016/j.acha.2022.08.003_br0010) 2017 Candes (10.1016/j.acha.2022.08.003_br0080) 2006; 59 Schaeffer (10.1016/j.acha.2022.08.003_br0470) 2020; 18 Ullah (10.1016/j.acha.2022.08.003_br0660) 2018 Rakotomamonjy (10.1016/j.acha.2022.08.003_br0580) 2013 Rosset (10.1016/j.acha.2022.08.003_br0570) 2007 Li (10.1016/j.acha.2022.08.003_br0320) 2019 Potts (10.1016/j.acha.2022.08.003_br0380) Constantine (10.1016/j.acha.2022.08.003_br0160) 2015; 302 Doostan (10.1016/j.acha.2022.08.003_br0190) 2011; 230 Marelli (10.1016/j.acha.2022.08.003_br0340) 2014 Cai (10.1016/j.acha.2022.08.003_br0070) 2009; 55 Donoho (10.1016/j.acha.2022.08.003_br0180) 2006; 52 Barron (10.1016/j.acha.2022.08.003_br0040) 1993; 39 Harris (10.1016/j.acha.2022.08.003_br0240) Rudi (10.1016/j.acha.2022.08.003_br0650) 2017 Du (10.1016/j.acha.2022.08.003_br0590) Figueiredo (10.1016/j.acha.2022.08.003_br0600) 2007 Rauhut (10.1016/j.acha.2022.08.003_br0430) 2012; 164 Adcock (10.1016/j.acha.2022.08.003_br0020) 2018; 18 Yang (10.1016/j.acha.2022.08.003_br0520) 2012; 25 van Den Berg (10.1016/j.acha.2022.08.003_br0680) 2009 Szabó (10.1016/j.acha.2022.08.003_br0670) 2019 Constantine (10.1016/j.acha.2022.08.003_br0150) 2017; 326 Rahimi (10.1016/j.acha.2022.08.003_br0410) 2008 Sutherland (10.1016/j.acha.2022.08.003_br0500) Rahimi (10.1016/j.acha.2022.08.003_br0400) 2008 Lim (10.1016/j.acha.2022.08.003_br0560) 2015 Maass (10.1016/j.acha.2022.08.003_br0330) 2004; 69 Rahimi (10.1016/j.acha.2022.08.003_br0420) 2007 10.1016/j.acha.2022.08.003_br0210 Yen (10.1016/j.acha.2022.08.003_br0530) 2014 Du (10.1016/j.acha.2022.08.003_br0200) Sriperumbudur (10.1016/j.acha.2022.08.003_br0490) Block (10.1016/j.acha.2022.08.003_br0060) 1962; 34 Candes (10.1016/j.acha.2022.08.003_br0090) 2006; 52 Li (10.1016/j.acha.2022.08.003_br0310) Liu (10.1016/j.acha.2022.08.003_br0620) 2020 Mei (10.1016/j.acha.2022.08.003_br0350) Shi (10.1016/j.acha.2022.08.003_br0540) 2007 Arora (10.1016/j.acha.2022.08.003_br0030) 2019 Kuo (10.1016/j.acha.2022.08.003_br0290) 2010; 79 Potts (10.1016/j.acha.2022.08.003_br0390) Moosmann (10.1016/j.acha.2022.08.003_br0360) 2006 Schaeffer (10.1016/j.acha.2022.08.003_br0460) 2018; 78 Heisenberg (10.1016/j.acha.2022.08.003_br0250) 1927 Constantine (10.1016/j.acha.2022.08.003_br0140) 2014; 36 Saltelli (10.1016/j.acha.2022.08.003_br0450) 2008 Jones (10.1016/j.acha.2022.08.003_br0280) 1992; 20 Kandasamy (10.1016/j.acha.2022.08.003_br0610) 2016 Chkifa (10.1016/j.acha.2022.08.003_br0130) 2018; 311 Guo (10.1016/j.acha.2022.08.003_br0550) 2012 Jacot (10.1016/j.acha.2022.08.003_br0270) Rauhut (10.1016/j.acha.2022.08.003_br0440) 2016; 40 |
| References_xml | – year: 2015 ident: br0500 article-title: On the error of random Fourier features – volume: 34 start-page: 123 year: 1962 ident: br0060 article-title: The perceptron: a model for brain functioning. I publication-title: Rev. Mod. Phys. – start-page: 197 year: 2017 end-page: 207 ident: br0630 article-title: Group sparse additive machine publication-title: Proceedings of the 31st International Conference on Neural Information Processing Systems – volume: 36 start-page: A1500 year: 2014 end-page: A1524 ident: br0140 article-title: Active subspace methods in theory and practice: applications to Kriging surfaces publication-title: SIAM J. Sci. Comput. – start-page: 1009 year: 2009 end-page: 1030 ident: br0640 article-title: Sparse additive models publication-title: J. R. Stat. Soc., Ser. B, Stat. Methodol. – start-page: 890 year: 2009 end-page: 912 ident: br0680 article-title: Probing the Pareto frontier for basis pursuit solutions publication-title: SIAM J. Sci. Comput. – start-page: 1 year: 2007 end-page: 5 ident: br0540 article-title: Detecting disease-causing genes by LASSO-Patternsearch algorithm publication-title: BMC Proc. – start-page: 35 year: 2017 end-page: 3225 ident: br0650 article-title: Generalization properties of learning with random features publication-title: Proceedings of the 31st International Conference on Neural Information Processing Systems – volume: 12 start-page: 229 year: 2012 end-page: 262 ident: br0220 article-title: Learning functions of few arbitrary linear parameters in high dimensions publication-title: Found. Comput. Math. – volume: 52 start-page: 5406 year: 2006 end-page: 5425 ident: br0090 article-title: Near-optimal signal recovery from random projections: universal encoding strategies? publication-title: IEEE Trans. Inf. Theory – start-page: 627 year: 2015 end-page: 654 ident: br0560 article-title: Learning interactions via hierarchical group-lasso regularization publication-title: J. Comput. Graph. Stat. – volume: 302 start-page: 1 year: 2015 end-page: 20 ident: br0160 article-title: Exploiting active subspaces to quantify uncertainty in the numerical simulation of the HyShot II scramjet publication-title: J. Comput. Phys. – start-page: 2554 year: 2014 end-page: 2563 ident: br0340 article-title: UQlab: a framework for uncertainty quantification in Matlab publication-title: Vulnerability, Uncertainty, and Risk: Quantification, Mitigation, and Management – year: 2008 ident: br0450 article-title: Global Sensitivity Analysis: the Primer – volume: 25 start-page: 476 year: 2012 end-page: 484 ident: br0520 article-title: Nyström method vs random Fourier features: a theoretical and empirical comparison publication-title: Adv. Neural Inf. Process. Syst. – year: 2019 ident: br0380 article-title: Approximation of high-dimensional periodic functions with Fourier-based methods – start-page: 1313 year: 2008 end-page: 1320 ident: br0410 article-title: Weighted sums of random kitchen sinks: replacing minimization with randomization in learning publication-title: Adv. Neural Inf. Process. Syst. – year: 2019 ident: br0390 article-title: Learning multivariate functions with low-dimensional structures using polynomial bases – start-page: 1 year: 2007 end-page: 10 ident: br0420 article-title: Random features for large-scale kernel machines publication-title: NIPS – volume: 20 start-page: 608 year: 1992 end-page: 613 ident: br0280 article-title: A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training publication-title: Ann. Stat. – volume: 69 start-page: 593 year: 2004 end-page: 616 ident: br0330 article-title: On the computational power of circuits of spiking neurons publication-title: J. Comput. Syst. Sci. – start-page: 2456 year: 2014 end-page: 2464 ident: br0530 article-title: Sparse random feature algorithm as coordinate descent in Hilbert space publication-title: Adv. Neural Inf. Process. Syst. – volume: 59 start-page: 1207 year: 2006 end-page: 1223 ident: br0080 article-title: Stable signal recovery from incomplete and inaccurate measurements publication-title: Commun. Pure Appl. Math. – year: 2019 ident: br0240 article-title: Additive function approximation in the brain – year: 2018 ident: br0310 article-title: Learning overparameterized neural networks via stochastic gradient descent on structured data – reference: D. Dua, C. Graff, UCI machine learning repository, 2017. – volume: 33 start-page: 125 year: 2011 end-page: 143 ident: br0170 article-title: Approximation of functions of few variables in high dimensions publication-title: Constr. Approx. – volume: 230 start-page: 3015 year: 2011 end-page: 3034 ident: br0190 article-title: A non-adapted sparse approximation of PDEs with stochastic inputs publication-title: J. Comput. Phys. – year: 2020 ident: br0590 article-title: Few-shot learning via learning the representation, provably – volume: 164 start-page: 517 year: 2012 end-page: 533 ident: br0430 article-title: Sparse Legendre expansions via l1-minimization publication-title: J. Approx. Theory – year: 2020 ident: br0350 article-title: The generalization error of random features regression: precise asymptotics and double descent curve – start-page: 827 year: 2019 end-page: 836 ident: br0670 article-title: On kernel derivative approximation with random Fourier features publication-title: Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics – year: 2006 ident: br0360 article-title: Randomized clustering forests for building fast and discriminative visual vocabularies publication-title: NIPS – volume: 18 start-page: 1435 year: 2020 end-page: 1461 ident: br0470 article-title: Extracting structured dynamical systems using sparse optimization with very few samples publication-title: Multiscale Model. Simul. – volume: 40 start-page: 3 year: 2016 end-page: 351 ident: br0440 article-title: Interpolation via weighted l1 minimization publication-title: Appl. Comput. Harmon. Anal. – start-page: 93 year: 2017 end-page: 124 ident: br0010 article-title: Compressed sensing approaches for polynomial approximation of high-dimensional functions publication-title: Compressed Sensing and its Applications – year: 2018 ident: br0200 article-title: Gradient descent provably optimizes over-parameterized neural networks – start-page: 69 year: 2016 end-page: 78 ident: br0610 article-title: Additive approximations in high dimensional nonparametric regression via the SALSA publication-title: International Conference on Machine Learning – volume: 55 start-page: 3388 year: 2009 end-page: 3397 ident: br0070 article-title: On recovery of sparse signals via publication-title: IEEE Trans. Inf. Theory – year: 2013 ident: br0230 article-title: A Mathematical Introduction to Compressive Sensing – start-page: 7322 year: 2018 end-page: 7332 ident: br0660 article-title: Streaming kernel PCA with publication-title: Features Proceedings of the 32nd International Conference on Neural Information Processing System – start-page: 3905 year: 2019 end-page: 3914 ident: br0320 article-title: Towards a unified analysis of random Fourier features publication-title: International Conference on Machine Learning – volume: 78 start-page: 3279 year: 2018 end-page: 3295 ident: br0460 article-title: Extracting sparse high-dimensional dynamics from limited data publication-title: SIAM J. Appl. Math. – start-page: 555 year: 2008 end-page: 561 ident: br0400 article-title: Uniform approximation of functions with random bases publication-title: 2008 46th Annual Allerton Conference on Communication, Control, and Computing – volume: 326 start-page: 402 year: 2017 end-page: 404 ident: br0150 article-title: A near-stationary subspace for ridge approximation publication-title: Comput. Methods Appl. Mech. Eng. – volume: 79 start-page: 953 year: 2010 end-page: 966 ident: br0290 article-title: On decompositions of multivariate functions publication-title: Math. Comput. – start-page: 586 year: 2007 end-page: 597 ident: br0600 article-title: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems publication-title: IEEE J. Sel. Top. Signal Process. – volume: 18 start-page: 661 year: 2018 end-page: 701 ident: br0020 article-title: Infinite-dimensional compressed sensing and function interpolation publication-title: Found. Comput. Math. – start-page: 544 year: 2007 end-page: 558 ident: br0570 article-title: l-1 regularization in infinite dimensional feature spaces publication-title: International Conference on Computational Learning Theory – start-page: 322 year: 2019 end-page: 332 ident: br0030 article-title: Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks publication-title: International Conference on Machine Learning – start-page: 43 year: 2013 end-page: 66 ident: br0580 article-title: Learning with infinitely many features publication-title: Machine Learning – volume: 311 start-page: 1415 year: 2018 end-page: 1450 ident: br0130 article-title: Polynomial approximation via compressed sensing of high-dimensional functions on lower sets publication-title: Math. Comput. – volume: 39 start-page: 930 year: 1993 end-page: 945 ident: br0040 article-title: Universal approximation bounds for superpositions of a sigmoidal function publication-title: IEEE Trans. Inf. Theory – start-page: 6194 year: 2020 end-page: 6204 ident: br0620 article-title: Sparse shrunk additive models publication-title: International Conference on Machine Learning – volume: 52 start-page: 1289 year: 2006 end-page: 1306 ident: br0180 article-title: Compressed sensing publication-title: IEEE Trans. Inf. Theory – start-page: 161 year: 2012 end-page: 170 ident: br0120 article-title: Efficient kernel clustering using random Fourier features publication-title: 2012 IEEE 12th International Conference on Data Mining – year: 2015 ident: br0490 article-title: Optimal rates for random Fourier features – start-page: 567 year: 2012 end-page: 577 ident: br0550 article-title: Bayesian lasso for semiparametric structural equation models publication-title: Biometrics – start-page: 172 year: 1927 end-page: 198 ident: br0250 article-title: About the descriptive content of quantum theoretical kinematics and mechanics publication-title: Z. Phys. – year: 2018 ident: br0270 article-title: Neural tangent kernel: convergence and generalization in neural networks – ident: 10.1016/j.acha.2022.08.003_br0350 – start-page: 69 year: 2016 ident: 10.1016/j.acha.2022.08.003_br0610 article-title: Additive approximations in high dimensional nonparametric regression via the SALSA – start-page: 1 year: 2007 ident: 10.1016/j.acha.2022.08.003_br0420 article-title: Random features for large-scale kernel machines – volume: 40 start-page: 3 issue: 2 year: 2016 ident: 10.1016/j.acha.2022.08.003_br0440 article-title: Interpolation via weighted l1 minimization publication-title: Appl. Comput. Harmon. Anal. doi: 10.1016/j.acha.2015.02.003 – start-page: 586 year: 2007 ident: 10.1016/j.acha.2022.08.003_br0600 article-title: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems publication-title: IEEE J. Sel. Top. Signal Process. doi: 10.1109/JSTSP.2007.910281 – volume: 25 start-page: 476 year: 2012 ident: 10.1016/j.acha.2022.08.003_br0520 article-title: Nyström method vs random Fourier features: a theoretical and empirical comparison publication-title: Adv. Neural Inf. Process. Syst. – volume: 36 start-page: A1500 issue: 4 year: 2014 ident: 10.1016/j.acha.2022.08.003_br0140 article-title: Active subspace methods in theory and practice: applications to Kriging surfaces publication-title: SIAM J. Sci. Comput. doi: 10.1137/130916138 – year: 2013 ident: 10.1016/j.acha.2022.08.003_br0230 – ident: 10.1016/j.acha.2022.08.003_br0500 – ident: 10.1016/j.acha.2022.08.003_br0310 – ident: 10.1016/j.acha.2022.08.003_br0490 – volume: 52 start-page: 1289 issue: 4 year: 2006 ident: 10.1016/j.acha.2022.08.003_br0180 article-title: Compressed sensing publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2006.871582 – start-page: 322 year: 2019 ident: 10.1016/j.acha.2022.08.003_br0030 article-title: Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks – volume: 326 start-page: 402 year: 2017 ident: 10.1016/j.acha.2022.08.003_br0150 article-title: A near-stationary subspace for ridge approximation publication-title: Comput. Methods Appl. Mech. Eng. doi: 10.1016/j.cma.2017.07.038 – start-page: 555 year: 2008 ident: 10.1016/j.acha.2022.08.003_br0400 article-title: Uniform approximation of functions with random bases – start-page: 6194 year: 2020 ident: 10.1016/j.acha.2022.08.003_br0620 article-title: Sparse shrunk additive models – start-page: 7322 year: 2018 ident: 10.1016/j.acha.2022.08.003_br0660 article-title: Streaming kernel PCA with O(n) random – volume: 302 start-page: 1 year: 2015 ident: 10.1016/j.acha.2022.08.003_br0160 article-title: Exploiting active subspaces to quantify uncertainty in the numerical simulation of the HyShot II scramjet publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2015.09.001 – start-page: 1009 year: 2009 ident: 10.1016/j.acha.2022.08.003_br0640 article-title: Sparse additive models publication-title: J. R. Stat. Soc., Ser. B, Stat. Methodol. doi: 10.1111/j.1467-9868.2009.00718.x – start-page: 2456 year: 2014 ident: 10.1016/j.acha.2022.08.003_br0530 article-title: Sparse random feature algorithm as coordinate descent in Hilbert space publication-title: Adv. Neural Inf. Process. Syst. – ident: 10.1016/j.acha.2022.08.003_br0270 – volume: 39 start-page: 930 issue: 3 year: 1993 ident: 10.1016/j.acha.2022.08.003_br0040 article-title: Universal approximation bounds for superpositions of a sigmoidal function publication-title: IEEE Trans. Inf. Theory doi: 10.1109/18.256500 – ident: 10.1016/j.acha.2022.08.003_br0200 – start-page: 2554 year: 2014 ident: 10.1016/j.acha.2022.08.003_br0340 article-title: UQlab: a framework for uncertainty quantification in Matlab – start-page: 3905 year: 2019 ident: 10.1016/j.acha.2022.08.003_br0320 article-title: Towards a unified analysis of random Fourier features – start-page: 567 year: 2012 ident: 10.1016/j.acha.2022.08.003_br0550 article-title: Bayesian lasso for semiparametric structural equation models publication-title: Biometrics doi: 10.1111/j.1541-0420.2012.01751.x – volume: 18 start-page: 661 issue: 3 year: 2018 ident: 10.1016/j.acha.2022.08.003_br0020 article-title: Infinite-dimensional compressed sensing and function interpolation publication-title: Found. Comput. Math. doi: 10.1007/s10208-017-9350-3 – volume: 52 start-page: 5406 issue: 12 year: 2006 ident: 10.1016/j.acha.2022.08.003_br0090 article-title: Near-optimal signal recovery from random projections: universal encoding strategies? publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2006.885507 – ident: 10.1016/j.acha.2022.08.003_br0210 – start-page: 161 year: 2012 ident: 10.1016/j.acha.2022.08.003_br0120 article-title: Efficient kernel clustering using random Fourier features – volume: 164 start-page: 517 issue: 5 year: 2012 ident: 10.1016/j.acha.2022.08.003_br0430 article-title: Sparse Legendre expansions via l1-minimization publication-title: J. Approx. Theory doi: 10.1016/j.jat.2012.01.008 – year: 2008 ident: 10.1016/j.acha.2022.08.003_br0450 – start-page: 172 year: 1927 ident: 10.1016/j.acha.2022.08.003_br0250 article-title: About the descriptive content of quantum theoretical kinematics and mechanics publication-title: Z. Phys. doi: 10.1007/BF01397280 – volume: 33 start-page: 125 issue: 1 year: 2011 ident: 10.1016/j.acha.2022.08.003_br0170 article-title: Approximation of functions of few variables in high dimensions publication-title: Constr. Approx. doi: 10.1007/s00365-010-9105-8 – start-page: 890 year: 2009 ident: 10.1016/j.acha.2022.08.003_br0680 article-title: Probing the Pareto frontier for basis pursuit solutions publication-title: SIAM J. Sci. Comput. doi: 10.1137/080714488 – volume: 55 start-page: 3388 issue: 7 year: 2009 ident: 10.1016/j.acha.2022.08.003_br0070 article-title: On recovery of sparse signals via ℓ1 minimization publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2009.2021377 – ident: 10.1016/j.acha.2022.08.003_br0240 – ident: 10.1016/j.acha.2022.08.003_br0390 – volume: 230 start-page: 3015 issue: 8 year: 2011 ident: 10.1016/j.acha.2022.08.003_br0190 article-title: A non-adapted sparse approximation of PDEs with stochastic inputs publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2011.01.002 – volume: 34 start-page: 123 issue: 1 year: 1962 ident: 10.1016/j.acha.2022.08.003_br0060 article-title: The perceptron: a model for brain functioning. I publication-title: Rev. Mod. Phys. doi: 10.1103/RevModPhys.34.123 – ident: 10.1016/j.acha.2022.08.003_br0380 – start-page: 43 year: 2013 ident: 10.1016/j.acha.2022.08.003_br0580 article-title: Learning with infinitely many features – year: 2006 ident: 10.1016/j.acha.2022.08.003_br0360 article-title: Randomized clustering forests for building fast and discriminative visual vocabularies – start-page: 1313 year: 2008 ident: 10.1016/j.acha.2022.08.003_br0410 article-title: Weighted sums of random kitchen sinks: replacing minimization with randomization in learning publication-title: Adv. Neural Inf. Process. Syst. – volume: 59 start-page: 1207 issue: 8 year: 2006 ident: 10.1016/j.acha.2022.08.003_br0080 article-title: Stable signal recovery from incomplete and inaccurate measurements publication-title: Commun. Pure Appl. Math. doi: 10.1002/cpa.20124 – volume: 311 start-page: 1415 issue: 87 year: 2018 ident: 10.1016/j.acha.2022.08.003_br0130 article-title: Polynomial approximation via compressed sensing of high-dimensional functions on lower sets publication-title: Math. Comput. – start-page: 827 year: 2019 ident: 10.1016/j.acha.2022.08.003_br0670 article-title: On kernel derivative approximation with random Fourier features – start-page: 93 year: 2017 ident: 10.1016/j.acha.2022.08.003_br0010 article-title: Compressed sensing approaches for polynomial approximation of high-dimensional functions – start-page: 1 year: 2007 ident: 10.1016/j.acha.2022.08.003_br0540 article-title: Detecting disease-causing genes by LASSO-Patternsearch algorithm publication-title: BMC Proc. – volume: 18 start-page: 1435 issue: 4 year: 2020 ident: 10.1016/j.acha.2022.08.003_br0470 article-title: Extracting structured dynamical systems using sparse optimization with very few samples publication-title: Multiscale Model. Simul. doi: 10.1137/18M1194730 – start-page: 544 year: 2007 ident: 10.1016/j.acha.2022.08.003_br0570 article-title: l-1 regularization in infinite dimensional feature spaces – start-page: 35 year: 2017 ident: 10.1016/j.acha.2022.08.003_br0650 article-title: Generalization properties of learning with random features – volume: 79 start-page: 953 issue: 270 year: 2010 ident: 10.1016/j.acha.2022.08.003_br0290 article-title: On decompositions of multivariate functions publication-title: Math. Comput. doi: 10.1090/S0025-5718-09-02319-9 – start-page: 197 year: 2017 ident: 10.1016/j.acha.2022.08.003_br0630 article-title: Group sparse additive machine – volume: 78 start-page: 3279 issue: 6 year: 2018 ident: 10.1016/j.acha.2022.08.003_br0460 article-title: Extracting sparse high-dimensional dynamics from limited data publication-title: SIAM J. Appl. Math. doi: 10.1137/18M116798X – volume: 12 start-page: 229 issue: 2 year: 2012 ident: 10.1016/j.acha.2022.08.003_br0220 article-title: Learning functions of few arbitrary linear parameters in high dimensions publication-title: Found. Comput. Math. doi: 10.1007/s10208-012-9115-y – volume: 69 start-page: 593 issue: 4 year: 2004 ident: 10.1016/j.acha.2022.08.003_br0330 article-title: On the computational power of circuits of spiking neurons publication-title: J. Comput. Syst. Sci. doi: 10.1016/j.jcss.2004.04.001 – start-page: 627 year: 2015 ident: 10.1016/j.acha.2022.08.003_br0560 article-title: Learning interactions via hierarchical group-lasso regularization publication-title: J. Comput. Graph. Stat. doi: 10.1080/10618600.2014.938812 – ident: 10.1016/j.acha.2022.08.003_br0590 – volume: 20 start-page: 608 issue: 1 year: 1992 ident: 10.1016/j.acha.2022.08.003_br0280 article-title: A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training publication-title: Ann. Stat. doi: 10.1214/aos/1176348546 |
| SSID | ssj0011459 |
| Score | 2.5503297 |
| Snippet | Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 310 |
| SubjectTerms | Compressive sensing Generalization error Random features Sparse optimization |
| Title | Generalization bounds for sparse random feature expansions |
| URI | https://dx.doi.org/10.1016/j.acha.2022.08.003 |
| Volume | 62 |
| WOSCitedRecordID | wos000870987200003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1096-603X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0011459 issn: 1063-5203 databaseCode: AIEXJ dateStart: 20211211 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Li9RAEG7GWQ96EJ-4vsjBWwhkupN0t7dFVlRwEWYX5hb6ia5jZpjHsnryp1v9SGdmlUUFL2HSTCdN1Ud1daXqK4Re8rJUtuSi0EqbohINLriltmjqmgnDDMay8s0m6MkJm834x9HoR18LczGnXccuL_nyv6oaxkDZrnT2L9SdHgoD8BuUDldQO1z_SPGRSDrWV-bS9U3ypAs5GI_V2uSwO-nF19waz-npOP5hu0phu56RNnqnsextud30UUPHde3b5ojIZzIYsbVjH_DWRi7mVnxP2RtTl5vvOrH4nU5800MB2tR3FY4Z3kPMe6m2bvjMbr_sBiYwuRKYSBUzewmdcAIlcPgtg1EzcYw3RVP6vsDJKkcbHcwqiamvYYcm4UvOL8Y_xCHOAYifHKMUxp6ctX_VHqn21K3DLQP7UqWa3EAHmNacjdHB0bvj2fv0JWpS-YZ7ad2x8CrkCF590--dmx2H5fQuuhNPGtlRQMg9NDLdfXR7h38S7j4k0t71A_RqHzlZQE4GyMkCcrKAnCwiJxuQ8xCdvTk-ff22iK01CkWqalNoWhksueTCEq6kJeBpS6xrl1FMDG5oo5XV3JUq25JIpRUF315TJYmYCJDXIzTuFp15jDImYRYT2mBaVkoxIZqGMUFLMOjMKHuIJr1MWhV55137k3nbJxiet06OrZNj63qiluQQ5WnOMrCuXPvvuhd1G_3G4A-2gIxr5j35x3lP0a0B78_QeLPamufoprrYfF6vXkQA_QTpgZPW |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Generalization+bounds+for+sparse+random+feature+expansions&rft.jtitle=Applied+and+computational+harmonic+analysis&rft.au=Hashemi%2C+Abolfazl&rft.au=Schaeffer%2C+Hayden&rft.au=Shi%2C+Robert&rft.au=Topcu%2C+Ufuk&rft.date=2023-01-01&rft.pub=Elsevier+Inc&rft.issn=1063-5203&rft.eissn=1096-603X&rft.volume=62&rft.spage=310&rft.epage=330&rft_id=info:doi/10.1016%2Fj.acha.2022.08.003&rft.externalDocID=S1063520322000653 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-5203&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-5203&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-5203&client=summon |