Stochastic Subgradient Method Converges on Tame Functions

This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity? We prove that the stochastic subgradient method, on any semialgebraic locally Lipschitz function, produces limit points that are all first-order stati...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Foundations of computational mathematics Jg. 20; H. 1; S. 119 - 154
Hauptverfasser: Davis, Damek, Drusvyatskiy, Dmitriy, Kakade, Sham, Lee, Jason D.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.02.2020
Springer Nature B.V
Schlagworte:
Applications of Mathematics
Computer Science
Convergence
Convexity
Economics
Linear and Multilinear Algebras
Machine learning
Math Applications in Computer Science
Mathematics
Mathematics and Statistics
Matrix Theory
Numerical Analysis
Queuing theory
Smoothness
Proximal
Subgradient
Tame
65K05
Semialgebraic
Stochastic subgradient method
65K10
90C15
Differential inclusion
34A60
Lyapunov function
ISSN:1615-3375, 1615-3383
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity? We prove that the stochastic subgradient method, on any semialgebraic locally Lipschitz function, produces limit points that are all first-order stationary. More generally, our result applies to any function with a Whitney stratifiable graph. In particular, this work endows the stochastic subgradient method, and its proximal extension, with rigorous convergence guarantees for a wide class of problems arising in data science—including all popular deep learning architectures.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1615-3375
1615-3383
DOI:10.1007/s10208-018-09409-5