A new efficient training strategy for deep neural networks by hybridization of artificial bee colony and limited–memory BFGS optimization algorithms

Working up with deep learning techniques requires profound understanding of the mechanisms underlying the optimization of the internal parameters of complex structures. The major factor limiting this understanding is that there exist only a few optimization methods such as gradient descent and Limit...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neurocomputing (Amsterdam) Ročník 266; s. 506 - 526
Hlavní autoři: Badem, Hasan, Basturk, Alper, Caliskan, Abdullah, Yuksel, Mehmet Emin
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier B.V 29.11.2017
Témata:
Artificial bee colony optimization algorithm
Deep learning
Deep neural network
Hybridization
L-BFGS
Stacked autoencoder network
Training strategy
Deep learning
Deep neural network
Stacked autoencoder network
Artificial bee colony optimization algorithm
L-BFGS
Hybridization
Training strategy
ISSN:0925-2312, 1872-8286
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Working up with deep learning techniques requires profound understanding of the mechanisms underlying the optimization of the internal parameters of complex structures. The major factor limiting this understanding is that there exist only a few optimization methods such as gradient descent and Limited–memory Broyden–Fletcher–Goldfarb–Shannon (L-BFGS) to find the best local minima of the problem space for these complex structures such as deep neural network (DNN). Therefore, in this paper, we represent a new training approach named hybrid artificial bee colony based training strategy (HABCbTS) to tune the parameters of a DNN structure, which includes one or more autoencoder layers cascaded to a softmax classification layer. In this strategy, a derivative-free optimization algorithm “ABC” is combined with a derivative-based algorithm “L-BFGS” to construct “HABC”, which is used in the HABCbTS. Detailed simulation results supported by statistical analysis show that the proposed training strategy results in better classification performance compared to the DNN classifier trained with the L-BFGS, ABC and modified ABC. The obtained classification results are also compared with the state-of-the-art classifiers, including MLP, SVM, KNN, DT and NB on 15 data sets with different dimensions and sizes.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2017.05.061