Accelerated First-Order Optimization Algorithms for Machine Learning

Numerical optimization serves as one of the pillars of machine learning. To meet the demands of big data applications, lots of efforts have been put on designing theoretically and practically fast algorithms. This article provides a comprehensive survey on accelerated first-order algorithms with a f...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the IEEE Ročník 108; číslo 11; s. 2067 - 2082
Hlavní autoři: Li, Huan, Fang, Cong, Lin, Zhouchen
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.11.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Acceleration
Algorithms
Approximation algorithms
Complexity theory
Computational geometry
Convergence
Convex analysis
Convex functions
convex optimization
Convexity
deterministic algorithms
First order algorithms
Machine learning
Machine learning algorithms
nonconvex optimization
Numerical analysis
Optimization
Optimization methods
stochastic algorithms
ISSN:0018-9219, 1558-2256
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Numerical optimization serves as one of the pillars of machine learning. To meet the demands of big data applications, lots of efforts have been put on designing theoretically and practically fast algorithms. This article provides a comprehensive survey on accelerated first-order algorithms with a focus on stochastic algorithms. Specifically, this article starts with reviewing the basic accelerated algorithms on deterministic convex optimization, then concentrates on their extensions to stochastic convex optimization, and at last introduces some recent developments on acceleration for nonconvex optimization.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9219
1558-2256
DOI:10.1109/JPROC.2020.3007634