Data-driven algorithm selection and tuning in optimization and signal processing

Machine learning algorithms typically rely on optimization subroutines and are well known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems?...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Annals of mathematics and artificial intelligence Ročník 89; číslo 7; s. 711 - 735
Hlavní autoři: De Loera, Jesús A., Haddock, Jamie, Ma, Anna, Needell, Deanna
Médium: Journal Article
Jazyk:angličtina
Vydáno: Cham Springer International Publishing 01.07.2021
Springer
Springer Nature B.V
Témata:
Algorithms
Artificial Intelligence
Complex Systems
Computer Science
Data mining
Data processing
Data science
Machine learning
Mathematics
Neural networks
Optimization
Performance enhancement
Signal processing
Subroutines
68T05
Neural networks
Algorithm selection
Automated machine learning
65K10
90C26
68T07
Compressed sensing
Hyperparameter tuning
ISSN:1012-2443, 1573-7470
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Machine learning algorithms typically rely on optimization subroutines and are well known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimization and signal processing algorithms. As a proof of concept, we use our approach to improve two popular data processing subroutines in data science: stochastic gradient descent and greedy methods in compressed sensing. We provide experimental results that demonstrate the answer is “yes”, machine learning algorithms do lead to more effective outcomes for optimization problems, and show the future potential for this research direction. In addition to our experimental work, we prove relevant Probably Approximately Correct (PAC) learning theorems for our problems of interest. More precisely, we show that there exists a learning algorithm that, with high probability, will select the algorithm that optimizes the average performance on an input set of problem instances with a given distribution.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1012-2443
1573-7470
DOI:10.1007/s10472-020-09717-z