How to tune the RBF SVM hyperparameters? An empirical evaluation of 18 search algorithms

SVM with an RBF kernel is usually one of the best classification algorithms for most data sets, but it is important to tune the two hyperparameters C and γ to the data itself. In general, the selection of the hyperparameters is a non-convex optimization problem and thus many algorithms have been pro...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:The Artificial intelligence review Ročník 54; číslo 6; s. 4771 - 4797
Hlavní autoři: Wainer, Jacques, Fonseca, Pablo
Médium: Journal Article
Jazyk:angličtina
Vydáno: Dordrecht Springer Netherlands 01.08.2021
Springer
Springer Nature B.V
Témata:
Algorithms
Analysis
Artificial Intelligence
Bayesian analysis
Binary data
Classification
Computation
Computational geometry
Computer Science
Convexity
Data
Datasets
Expenditures
Function words
Mathematical optimization
Methods
Optimization
Particle swarm optimization
Search algorithms
Searches and seizures
Simulated annealing
Trees
Non-convex optimization algorithms
Grid search
SVM
Hyperparameters
Random search
ISSN:0269-2821, 1573-7462
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:SVM with an RBF kernel is usually one of the best classification algorithms for most data sets, but it is important to tune the two hyperparameters C and γ to the data itself. In general, the selection of the hyperparameters is a non-convex optimization problem and thus many algorithms have been proposed to solve it, among them: grid search, random search, Bayesian optimization, simulated annealing, particle swarm optimization, Nelder Mead, and others. There have also been proposals to decouple the selection of γ and C . We empirically compare 18 of these proposed search algorithms (with different parameterizations for a total of 47 combinations) on 115 real-life binary data sets. We find (among other things) that trees of Parzen estimators and particle swarm optimization select better hyperparameters with only a slight increase in computation time with respect to a grid search with the same number of evaluations. We also find that spending too much computational effort searching the hyperparameters will not likely result in better performance for future data and that there are no significant differences among the different procedures to select the best set of hyperparameters when more than one is found by the search algorithms.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0269-2821
1573-7462
DOI:10.1007/s10462-021-10011-5