Random features and polynomial rules
Uložené v:
| Názov: | Random features and polynomial rules |
|---|---|
| Autori: | Fabian Aguirre-Lopez, Silvio Franz, Mauro Pastore |
| Zdroj: | SciPost Physics, Vol 18, Iss 1, p 039 (2025) |
| Publication Status: | Preprint |
| Informácie o vydavateľovi: | Stichting SciPost, 2025. |
| Rok vydania: | 2025 |
| Predmety: | FOS: Computer and information sciences, Computer Science - Machine Learning, Physics, QC1-999, Learning and adaptive systems in artificial intelligence, FOS: Physical sciences, Disordered Systems and Neural Networks (cond-mat.dis-nn), Condensed Matter - Disordered Systems and Neural Networks, Disordered systems (random Ising models, random Schrödinger operators, etc.) in equilibrium statistical mechanics, Artificial neural networks and deep learning, Machine Learning (cs.LG) |
| Popis: | Random features models play a distinguished role in the theory of deep learning, describing the behavior of neural networks close to their infinite-width limit. In this work, we present a thorough analysis of the generalization performance of random features models for generic supervised learning problems with Gaussian data. Our approach, built with tools from the statistical mechanics of disordered systems, maps the random features model to an equivalent polynomial model, and allows us to plot average generalization curves as functions of the two main control parameters of the problem: the number of random features NN and the size PP of the training set, both assumed to scale as powers in the input dimension DD. Our results extend the case of proportional scaling between NN, PP and DD. They are in accordance with rigorous bounds known for certain particular learning tasks and are in quantitative agreement with numerical experiments performed over many order of magnitudes of NN and PP. We find good agreement also far from the asymptotic limits where D\to ∞D→∞ and at least one between P/D^KP/DK, N/D^LN/DL remains finite. |
| Druh dokumentu: | Article |
| Popis súboru: | application/xml; application/pdf |
| ISSN: | 2542-4653 |
| DOI: | 10.21468/scipostphys.18.1.039 |
| DOI: | 10.48550/arxiv.2402.10164 |
| Prístupová URL adresa: | http://arxiv.org/abs/2402.10164 https://zbmath.org/7982799 https://doi.org/10.21468/scipostphys.18.1.039 https://doaj.org/article/04cae1a59a9b49168a087650d8e904b0 https://scipost.org/10.21468/SciPostPhys.18.1.039 https://doi.org/10.21468/SciPostPhys.18.1.039 https://hdl.handle.net/11587/552001 |
| Rights: | CC BY arXiv Non-Exclusive Distribution |
| Prístupové číslo: | edsair.doi.dedup.....c9c8442c64c34e7b2c48728a8ca4487d |
| Databáza: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstrakt: | Random features models play a distinguished role in the theory of deep learning, describing the behavior of neural networks close to their infinite-width limit. In this work, we present a thorough analysis of the generalization performance of random features models for generic supervised learning problems with Gaussian data. Our approach, built with tools from the statistical mechanics of disordered systems, maps the random features model to an equivalent polynomial model, and allows us to plot average generalization curves as functions of the two main control parameters of the problem: the number of random features NN and the size PP of the training set, both assumed to scale as powers in the input dimension DD. Our results extend the case of proportional scaling between NN, PP and DD. They are in accordance with rigorous bounds known for certain particular learning tasks and are in quantitative agreement with numerical experiments performed over many order of magnitudes of NN and PP. We find good agreement also far from the asymptotic limits where D\to ∞D→∞ and at least one between P/D^KP/DK, N/D^LN/DL remains finite. |
|---|---|
| ISSN: | 25424653 |
| DOI: | 10.21468/scipostphys.18.1.039 |