A New Correntropy-Based Conjugate Gradient Backpropagation Algorithm for Improving Training in Neural Networks

Mean square error (MSE) is the most prominent criterion in training neural networks and has been employed in numerous learning problems. In this paper, we suggest a group of novel robust information theoretic backpropagation (BP) methods, as correntropy-based conjugate gradient BP (CCG-BP). CCG-BP a...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transaction on neural networks and learning systems Ročník 29; číslo 12; s. 6252 - 6263
Hlavní autoři: Heravi, Ahmad Reza, Abed Hodtani, Ghosheh
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States IEEE 01.12.2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Artificial Intelligence
Artificial neural networks
Back propagation
Chaos theory
conjugate gradient (CG) descent
Conjugates
Convergence
correntropy
Humans
Information Storage and Retrieval
Information theory
mean square error (MSE) methods
Models, Neurological
Neural networks
Neural Networks (Computer)
Noise
Optimization
optimization methods
Pattern Recognition, Automated - methods
Robustness
Robustness (mathematics)
Search methods
Signal processing algorithms
Systems Theory
Training
ISSN:2162-237X, 2162-2388, 2162-2388
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Mean square error (MSE) is the most prominent criterion in training neural networks and has been employed in numerous learning problems. In this paper, we suggest a group of novel robust information theoretic backpropagation (BP) methods, as correntropy-based conjugate gradient BP (CCG-BP). CCG-BP algorithms converge faster than the common correntropy-based BP algorithms and have better performance than the common CG-BP algorithms based on MSE, especially in nonGaussian environments and in cases with impulsive noise or heavy-tailed distributions noise. In addition, a convergence analysis of this new type of method is particularly considered. Numerical results for several samples of function approximation, synthetic function estimation, and chaotic time series prediction illustrate that our new BP method is more robust than the MSE-based method in the sense of impulsive noise, especially when SNR is low.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2018.2827778