Convergence of a Batch Gradient Algorithm with Adaptive Momentum for Neural Networks

In this paper, a batch gradient algorithm with adaptive momentum is considered and a convergence theorem is presented when it is used for two-layer feedforward neural networks training. Simple but necessary sufficient conditions are offered to guarantee both weak and strong convergence. Compared wit...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Neural processing letters Jg. 34; H. 3; S. 221 - 228
Hauptverfasser: Shao, Hongmei, Xu, Dongpo, Zheng, Gaofeng
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Boston Springer US 01.12.2011
Springer
Springer Nature B.V
Schlagworte:
Adaptive algorithms
Applied sciences
Artificial Intelligence
Artificial neural networks
Complex Systems
Computational Intelligence
Computer Science
Computer science; control theory; systems
Connectionism. Neural networks
Convergence
Error functions
Exact sciences and technology
Momentum
Neural network
Gradient algorithm
Adaptive momentum
Convergence
Gradient
Adaptive algorithm
Error function
Adaptive method
Search algorithm
Batch process
Batch production
Feedforward
Quadratic function
ISSN:1370-4621, 1573-773X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, a batch gradient algorithm with adaptive momentum is considered and a convergence theorem is presented when it is used for two-layer feedforward neural networks training. Simple but necessary sufficient conditions are offered to guarantee both weak and strong convergence. Compared with existing general requirements, we do not restrict the error function to be quadratic or uniformly convex. A numerical example is supplied to illustrate the performance of the algorithm and support our theoretical finding.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1370-4621
1573-773X
DOI:10.1007/s11063-011-9193-x