SAGRAD: A Program for Neural Network Training with Simulated Annealing and the Conjugate Gradient Method

SAGRAD (Simulated Annealing GRADient), a Fortran 77 program for computing neural networks for classification using batch learning, is discussed. Neural network training in SAGRAD is based on a combination of simulated annealing and Møller's scaled conjugate gradient algorithm, the latter a vari...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of research of the National Institute of Standards and Technology Jg. 120; S. 113 - 128
Hauptverfasser: Bernal, Javier, Torres-Jimenez, Jose
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States National Institute of Standards and Technology 2015
[Gaithersburg, MD] : U.S. Dept. of Commerce, National Institute of Standards and Technology
Schlagworte:
batch learning
scaled conjugate gradient algorithm
simulated annealing
neural networks for classification
ISSN:2165-7254, 1044-677X, 2165-7254
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:SAGRAD (Simulated Annealing GRADient), a Fortran 77 program for computing neural networks for classification using batch learning, is discussed. Neural network training in SAGRAD is based on a combination of simulated annealing and Møller's scaled conjugate gradient algorithm, the latter a variation of the traditional conjugate gradient method, better suited for the nonquadratic nature of neural networks. Different aspects of the implementation of the training process in SAGRAD are discussed, such as the efficient computation of gradients and multiplication of vectors by Hessian matrices that are required by Møller's algorithm; the (re)initialization of weights with simulated annealing required to (re)start Møller's algorithm the first time and each time thereafter that it shows insufficient progress in reaching a possibly local minimum; and the use of simulated annealing when Møller's algorithm, after possibly making considerable progress, becomes stuck at a local minimum or flat area of weight space. Outlines of the scaled conjugate gradient algorithm, the simulated annealing procedure and the training process used in SAGRAD are presented together with results from running SAGRAD on two examples of training data.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:2165-7254
1044-677X
2165-7254
DOI:10.6028/jres.120.009