First-order optimization algorithms via inertial systems with Hessian driven damping

In a Hilbert space setting, for convex optimization, we analyze the convergence rate of a class of first-order algorithms involving inertial features. They can be interpreted as discrete time versions of inertial dynamics involving both viscous and Hessian-driven dampings. The geometrical damping dr...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Mathematical programming Ročník 193; číslo 1; s. 113 - 155
Hlavní autori:	Attouch, Hedy, Chbani, Zaki, Fadili, Jalal, Riahi, Hassan
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2022 Springer Springer Nature B.V Springer Verlag
Predmet:	Algorithms Analysis Calculus of Variations and Optimal Control; Optimization Combinatorics Computational geometry Convergence Convex analysis Convexity Damping First order algorithms Full Length Paper Hilbert space Inventory control Machine Learning Mathematical analysis Mathematical and Computational Physics Mathematical Methods in Physics Mathematical optimization Mathematics Mathematics and Statistics Mathematics of Computing Numerical Analysis Optimization Optimization and Control Regularization Statistics Theoretical 65K05 Time rescaling 37N40 65B99 49M30 Nesterov accelerated gradient method Ravine method 90C25 Inertial optimization algorithms 65K10 90B50 46N10 Hessian driven damping inertial optimization algorithms
ISSN:	0025-5610, 1436-4646
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Popis
Shrnutí:	In a Hilbert space setting, for convex optimization, we analyze the convergence rate of a class of first-order algorithms involving inertial features. They can be interpreted as discrete time versions of inertial dynamics involving both viscous and Hessian-driven dampings. The geometrical damping driven by the Hessian intervenes in the dynamics in the form ∇ 2 f ( x ( t ) ) x ˙ ( t ) . By treating this term as the time derivative of ∇ f ( x ( t ) ) , this gives, in discretized form, first-order algorithms in time and space. In addition to the convergence properties attached to Nesterov-type accelerated gradient methods, the algorithms thus obtained are new and show a rapid convergence towards zero of the gradients. On the basis of a regularization technique using the Moreau envelope, we extend these methods to non-smooth convex functions with extended real values. The introduction of time scale factors makes it possible to further accelerate these algorithms. We also report numerical results on structured problems to support our theoretical findings.
Bibliografia:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ISSN:	0025-5610 1436-4646
DOI:	10.1007/s10107-020-01591-1