Data-driven algorithm selection and tuning in optimization and signal processing

Machine learning algorithms typically rely on optimization subroutines and are well known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems?...

Full description

Saved in:
Bibliographic Details
Published in:Annals of mathematics and artificial intelligence Vol. 89; no. 7; pp. 711 - 735
Main Authors: De Loera, Jesús A., Haddock, Jamie, Ma, Anna, Needell, Deanna
Format: Journal Article
Language:English
Published: Cham Springer International Publishing 01.07.2021
Springer
Springer Nature B.V
Subjects:
Algorithms
Artificial Intelligence
Complex Systems
Computer Science
Data mining
Data processing
Data science
Machine learning
Mathematics
Neural networks
Optimization
Performance enhancement
Signal processing
Subroutines
68T05
Neural networks
Algorithm selection
Automated machine learning
65K10
90C26
68T07
Compressed sensing
Hyperparameter tuning
ISSN:1012-2443, 1573-7470
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Machine learning algorithms typically rely on optimization subroutines and are well known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimization and signal processing algorithms. As a proof of concept, we use our approach to improve two popular data processing subroutines in data science: stochastic gradient descent and greedy methods in compressed sensing. We provide experimental results that demonstrate the answer is “yes”, machine learning algorithms do lead to more effective outcomes for optimization problems, and show the future potential for this research direction. In addition to our experimental work, we prove relevant Probably Approximately Correct (PAC) learning theorems for our problems of interest. More precisely, we show that there exists a learning algorithm that, with high probability, will select the algorithm that optimizes the average performance on an input set of problem instances with a given distribution.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1012-2443
1573-7470
DOI:10.1007/s10472-020-09717-z