Convergence of a modified gradient-based learning algorithm with penalty for single-hidden-layer feed-forward networks

Based on a novel algorithm, known as the upper-layer-solution-aware (USA), a new algorithm, in which the penalty method is introduced into the empirical risk, is studied for training feed-forward neural networks in this paper, named as USA with penalty. Both theoretical analysis and numerical result...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neural computing & applications Ročník 32; číslo 7; s. 2445 - 2456
Hlavní autoři: Wang, Jian, Zhang, Bingjie, Sang, Zhaoyang, Liu, Yusong, Wu, Shujun, Miao, Quan
Médium: Journal Article
Jazyk:angličtina
Vydáno: London Springer London 01.04.2020
Springer Nature B.V
Témata:
Algorithms
Artificial Intelligence
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Convergence
Data Mining and Knowledge Discovery
Empirical analysis
Error functions
Image Processing and Computer Vision
Machine learning
Neural networks
Original Article
Probability and Statistics in Computer Science
Training
United States > US
Gradient
Neural networks
Penalty
Monotonicity
Convergence
ISSN:0941-0643, 1433-3058
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Based on a novel algorithm, known as the upper-layer-solution-aware (USA), a new algorithm, in which the penalty method is introduced into the empirical risk, is studied for training feed-forward neural networks in this paper, named as USA with penalty. Both theoretical analysis and numerical results show that it can control the magnitude of weights of the networks. Moreover, the deterministic theoretical analysis of the new algorithm is proved. The monotonicity of the empirical risk with penalty term is guaranteed in the training procedure. The weak and strong convergence results indicate that the gradient of the total error function with respect to weights tends to zero, and the weight sequence goes to a fixed point when the iterations approach positive infinity. Numerical experiment has been implemented and effectively verifies the proved theoretical results.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-018-3748-y