numerical analysis near singularities in rbf networks: Numerical analysis near singularities in RBF networks
Uloženo v:
| Název: | numerical analysis near singularities in rbf networks: Numerical analysis near singularities in RBF networks |
|---|---|
| Autoři: | Guo, Weili, Ong, Yew-Soon, Hervas, Jaime Rubio, Zhao, Junsheng, Zhang, Kanjian, Wang, Hai, Wei, Haikun |
| Přispěvatelé: | School of Computer Science and Engineering |
| Zdroj: | Journal of Machine Learning Research. 19(1):1-39 |
| Informace o vydavateli: | Microtome Publishing, Brookline, MA, 2018. |
| Rok vydání: | 2018 |
| Témata: | Engineering::Computer science and engineering [DRNTU], Neural nets and related approaches to inference from stochastic processes, learning dynamics, Singularity, numerical analysis, RBF networks, deep learning, Numerical aspects of recurrence relations, Statistical aspects of information-theoretic topics, singularity, RBF Networks, Artificial neural networks and deep learning |
| Popis: | Summary: The existence of singularities often affects the learning dynamics in feedforward neural networks. In this paper, based on theoretical analysis results, we numerically analyze the learning dynamics of radial basis function (RBF) networks near singularities to understand to what extent singularities influence the learning dynamics. First, we show the explicit expression of the Fisher information matrix for RBF networks. Second, we demonstrate through numerical simulations that the singularities have a significant impact on the learning dynamics of RBF networks. Our results show that overlap singularities mainly have influence on the low dimensional RBF networks and elimination singularities have a more significant impact to the learning processes than overlap singularities in both low and high dimensional RBF networks, whereas the plateau phenomena are mainly caused by the elimination singularities. The results can also be the foundation to investigate the singular learning dynamics in deep feedforward neural networks. |
| Druh dokumentu: | Article |
| Popis souboru: | application/xml; application/pdf |
| ISSN: | 1532-4435 |
| DOI: | 10.5555/3291125.3291126 |
| Přístupová URL adresa: | https://zbmath.org/6982292 https://dr.ntu.edu.sg/handle/10220/46400 https://jmlr.org/papers/v19/16-210.html https://dblp.uni-trier.de/db/journals/jmlr/jmlr19.html#GuoWOHZWZ18 https://jmlr.org/papers/volume19/16-210/16-210.pdf https://dl.acm.org/doi/10.5555/3291125.3291126 https://jmlr.csail.mit.edu/papers/v19/16-210.html http://www.jmlr.org/papers/volume19/16-210/16-210.pdf https://hdl.handle.net/10356/89770 http://hdl.handle.net/10220/46400 |
| Přístupové číslo: | edsair.dedup.wf.002..c2d46e86de8e6ec291279c67bdb059b7 |
| Databáze: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstrakt: | Summary: The existence of singularities often affects the learning dynamics in feedforward neural networks. In this paper, based on theoretical analysis results, we numerically analyze the learning dynamics of radial basis function (RBF) networks near singularities to understand to what extent singularities influence the learning dynamics. First, we show the explicit expression of the Fisher information matrix for RBF networks. Second, we demonstrate through numerical simulations that the singularities have a significant impact on the learning dynamics of RBF networks. Our results show that overlap singularities mainly have influence on the low dimensional RBF networks and elimination singularities have a more significant impact to the learning processes than overlap singularities in both low and high dimensional RBF networks, whereas the plateau phenomena are mainly caused by the elimination singularities. The results can also be the foundation to investigate the singular learning dynamics in deep feedforward neural networks. |
|---|---|
| ISSN: | 15324435 |
| DOI: | 10.5555/3291125.3291126 |