An elementary approach to tight worst case complexity analysis of gradient based methods

This work presents a novel analysis that allows to achieve tight complexity bounds of gradient-based methods for convex optimization. We start by identifying some of the pitfalls rooted in the classical complexity analysis of the gradient descent method, and show how they can be remedied. Our method...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Mathematical programming Ročník 201; číslo 1-2; s. 63 - 96
Hlavní autoři:	Teboulle, Marc, Vaisbourd, Yakov
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2023 Springer
Témata:	Calculus of Variations and Optimal Control; Optimization Combinatorics Full Length Paper Mathematical and Computational Physics Mathematical Methods in Physics Mathematics Mathematics and Statistics Mathematics of Computing Numerical Analysis Theoretical Convex minimization Gradient descent 68Q25 Performance estimation problem Proximal schemes 90C30 Composite minimization Global rate of convergence 90C25 90C60 Worst-case complexity analysis
ISSN:	0025-5610, 1436-4646
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	This work presents a novel analysis that allows to achieve tight complexity bounds of gradient-based methods for convex optimization. We start by identifying some of the pitfalls rooted in the classical complexity analysis of the gradient descent method, and show how they can be remedied. Our methodology hinges on elementary and direct arguments in the spirit of the classical analysis. It allows us to establish some new (and reproduce known) tight complexity results for several fundamental algorithms including, gradient descent, proximal point and proximal gradient methods which previously could be proven only through computer-assisted convergence proof arguments.
ISSN:	0025-5610 1436-4646
DOI:	10.1007/s10107-022-01899-0