Convergence of Stochastic Proximal Gradient Algorithm

We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under fu...

Full description

Saved in:
Bibliographic Details
Published in:Applied mathematics & optimization Vol. 82; no. 3; pp. 891 - 917
Main Authors: Rosasco, Lorenzo, Villa, Silvia, Vũ, Bằng Công
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2020
Springer Nature B.V
Subjects:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Control
Convergence
Convexity
Machine learning
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Numerical and Computational Physics
Simulation
Systems Theory
Theoretical
Forward–backward splitting algorithm
Stochastic optimization
Proximal methods
ISSN:0095-4616, 1432-0606
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0095-4616
1432-0606
DOI:10.1007/s00245-019-09617-7