Convergence of a Batch Gradient Algorithm with Adaptive Momentum for Neural Networks

In this paper, a batch gradient algorithm with adaptive momentum is considered and a convergence theorem is presented when it is used for two-layer feedforward neural networks training. Simple but necessary sufficient conditions are offered to guarantee both weak and strong convergence. Compared wit...

Full description

Saved in:
Bibliographic Details
Published in:Neural processing letters Vol. 34; no. 3; pp. 221 - 228
Main Authors: Shao, Hongmei, Xu, Dongpo, Zheng, Gaofeng
Format: Journal Article
Language:English
Published: Boston Springer US 01.12.2011
Springer
Springer Nature B.V
Subjects:
Adaptive algorithms
Applied sciences
Artificial Intelligence
Artificial neural networks
Complex Systems
Computational Intelligence
Computer Science
Computer science; control theory; systems
Connectionism. Neural networks
Convergence
Error functions
Exact sciences and technology
Momentum
Neural network
Gradient algorithm
Adaptive momentum
Convergence
Gradient
Adaptive algorithm
Error function
Adaptive method
Search algorithm
Batch process
Batch production
Feedforward
Quadratic function
ISSN:1370-4621, 1573-773X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, a batch gradient algorithm with adaptive momentum is considered and a convergence theorem is presented when it is used for two-layer feedforward neural networks training. Simple but necessary sufficient conditions are offered to guarantee both weak and strong convergence. Compared with existing general requirements, we do not restrict the error function to be quadratic or uniformly convex. A numerical example is supplied to illustrate the performance of the algorithm and support our theoretical finding.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1370-4621
1573-773X
DOI:10.1007/s11063-011-9193-x