Optimization of feedforward neural networks

This paper presents some novel approaches in the design of neural networks with one or two hidden layers trained by the backpropagation algorithm. First, hybrid neural networks that have different activation functions for different layers in fully connected feedforward neural networks are introduced...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Engineering applications of artificial intelligence Ročník 9; číslo 2; s. 109 - 119
Hlavní autoři: Han, Jun, Moraga, Claudio, Sinne, Stefan
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Ltd 1996
Témata:
Feedforward neural networks
parametric nets
soft computing
soft computing
parametric nets
Feedforward neural networks
ISSN:0952-1976, 1873-6769
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper presents some novel approaches in the design of neural networks with one or two hidden layers trained by the backpropagation algorithm. First, hybrid neural networks that have different activation functions for different layers in fully connected feedforward neural networks are introduced. Second, a variant sigmoid function with three parameters is discussed. The parameters are the dynamic range, symmetry and slope of the function respectively. It is illustrated how these parameters influence the speed of backpropagation learning, and a parametric feedforward network with different parameter configurations in different layers is introduced. By regulating and modifying parameter configurations of the sigmoid function in different layers the error signal problem, oscillation problem and asymmetrical input problem can be reduced. Furthermore, hybrid optimization methods for the dynamic parameters are introduced: Genetic algorithms are used to optimize the initial parameter configuration. The dynamic parameters are adjusted on-line using gradient descent methods. Sequential adjustment algorithms are derived in order to avoid the moving target problem and to increase the stability of the gradient descent methods. The new schemes have advantages in both convergence speed and generalization capability. Experimental results on the two-spirals problem are provided.
ISSN:0952-1976
1873-6769
DOI:10.1016/0952-1976(95)00001-1