Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization

The performance of machine learning algorithms are affected by several factors, some of these factors are related to data quantity, quality, or its features. Another element is the choice of an appropriate algorithm to solve the problem and one major influence is the parameter configuration based on...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2021 IEEE Congress on Evolutionary Computation (CEC) S. 1551 - 1559
Hauptverfasser: Alibrahim, Hussain, Ludwig, Simone A.
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.01.2021
Schlagworte:
Bayes methods
Bayesian
Genetic Algorithm
Grid Search
Hyperparmeter optimization
Machine learning algorithms
Neural networks
Prediction algorithms
Search problems
Time measurement
Training
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The performance of machine learning algorithms are affected by several factors, some of these factors are related to data quantity, quality, or its features. Another element is the choice of an appropriate algorithm to solve the problem and one major influence is the parameter configuration based on the problem specification. Parameters in machine learning can be classified in two types: (1) model parameters that are internal, configurable, and its value can be estimated from data such as weights of a deep neural network; and (2) hyperparameters, which are external and its values can not be estimated from data such as the learning rate for the training of a neural network. Hyperparameter values may be specified by a practitioner or using a heuristic, or parameter values obtained from other problems can be used etc., however, the best values of these parameters are identified when the algorithm has the highest accuracy, and these could be achieved by tuning the parameters. The main goal of this paper is to conduct a comparison study between different algorithms that are used in the optimization process in order to find the best hyperparameter values for the neural network. The algorithms applied are grid search algorithm, bayesian algorithm, and genetic algorithm. Different evaluation measures are used to conduct this comparison such as accuracy and running time.
DOI:10.1109/CEC45853.2021.9504761