Convergence of Stochastic Proximal Gradient Algorithm

We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under fu...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Applied mathematics & optimization Jg. 82; H. 3; S. 891 - 917
Hauptverfasser: Rosasco, Lorenzo, Villa, Silvia, Vũ, Bằng Công
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.12.2020
Springer Nature B.V
Schlagworte:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Control
Convergence
Convexity
Machine learning
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Numerical and Computational Physics
Simulation
Systems Theory
Theoretical
Forward–backward splitting algorithm
Stochastic optimization
Proximal methods
ISSN:0095-4616, 1432-0606
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0095-4616
1432-0606
DOI:10.1007/s00245-019-09617-7