Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches

This paper proposes a method to find the hyperparameter tuning for a deep neural network by using a univariate dynamic encoding algorithm for searches. Optimizing hyperparameters for such a neural network is difficult because the neural network that has several parameters to configure; furthermore,...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Knowledge-based systems Ročník 178; s. 74 - 83
Hlavní autor: Yoo, YoungJun
Médium: Journal Article
Jazyk:angličtina
Vydáno: Amsterdam Elsevier B.V 15.08.2019
Elsevier Science Ltd
Témata:
Algorithms
Artificial neural networks
Autoencoder
Averages
Computer simulation
Convergence
Convolution
Convolution neural network
Deep neural network
Encoding
Function words
Genetic algorithms
Genetics
Gradient-free optimization
Hyperparameter optimization
Networks
Neural networks
Optimization
Searching
Simulated annealing
Hyperparameter optimization
Deep neural network
Convolution neural network
Autoencoder
Gradient-free optimization
ISSN:0950-7051, 1872-7409
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper proposes a method to find the hyperparameter tuning for a deep neural network by using a univariate dynamic encoding algorithm for searches. Optimizing hyperparameters for such a neural network is difficult because the neural network that has several parameters to configure; furthermore, the training speed for such a network is slow. The proposed method was tested for two neural network models; an autoencoder and a convolution neural network with the Modified National Institute of Standards and Technology (MNIST) dataset. To optimize hyperparameters with the proposed method, the cost functions were selected as the average of the difference between the decoded value and the original image for the autoencoder, and the inverse of the evaluation accuracy for the convolution neural network. The hyperparameters were optimized using the proposed method with fast convergence speed and few computational resources, and the results were compared with those of the other considered optimization algorithms (namely, simulated annealing, genetic algorithm, and particle swarm algorithm) to show the effectiveness of the proposed methodology. •An optimization method for hyper-parameters for a deep neural network.•Performing optimization of the network using a univariate dynamic encoding algorithm for searches.•Validation of the proposed method with two neural network model with MNIST data set.•Fast convergence speed and a small computational amount to optimize hyper-parameter of the network.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0950-7051
1872-7409
DOI:10.1016/j.knosys.2019.04.019