The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions

In this paper, we study the convergence rate of the gradient (or steepest descent) method with fixed step lengths for finding a stationary point of an L -smooth function. We establish a new convergence rate, and show that the bound may be exact in some cases, in particular when all step lengths lie...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Optimization letters Jg. 16; H. 6; S. 1649 - 1661
Hauptverfasser: Abbaszadehpeivasti, Hadi, de Klerk, Etienne, Zamani, Moslem
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Berlin/Heidelberg Springer Berlin Heidelberg 01.07.2022
Schlagworte:
Computational Intelligence
Mathematics
Mathematics and Statistics
Numerical and Computational Physics
Operations Research/Decision Theory
Optimization
Original Paper
Simulation
smooth optimization
Performance estimation problem
Semidefinite programming
Gradient method
ISSN:1862-4472, 1862-4480
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, we study the convergence rate of the gradient (or steepest descent) method with fixed step lengths for finding a stationary point of an L -smooth function. We establish a new convergence rate, and show that the bound may be exact in some cases, in particular when all step lengths lie in the interval (0, 1/ L ]. In addition, we derive an optimal step length with respect to the new bound.
ISSN:1862-4472
1862-4480
DOI:10.1007/s11590-021-01821-1