New analysis of linear convergence of gradient-type methods via unifying error bound conditions

This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other optimality measures, we define a group of abstract EB conditions, and then analyze th...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Mathematical programming Ročník 180; číslo 1-2; s. 371 - 416
Hlavný autor: Zhang, Hui
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2020
Springer Nature B.V
Predmet:
Algorithms
Ascent
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Convergence
Convexity
Full Length Paper
Linearization
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Measurement methods
Numerical Analysis
Optimization
Theoretical
Error bound
Proximal point algorithm
Dual objective function
Residual measure operator
Gradient descent
Linear convergence
Proximal alternating linearized minimization
90C25
49M29
65K10
Forward–backward splitting algorithm
Nesterov’s acceleration
90C60
ISSN:0025-5610, 1436-4646
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other optimality measures, we define a group of abstract EB conditions, and then analyze the interplay between them; on the other hand, by using this operator as an ascent direction, we propose an abstract gradient-type method, and then derive EB conditions that are necessary and sufficient for its linear convergence. The former provides a unified framework that not only allows us to find new connections between many existing EB conditions, but also paves a way to construct new ones. The latter allows us to claim the weakest conditions guaranteeing linear convergence for a number of fundamental algorithms, including the gradient method, the proximal point algorithm, and the forward–backward splitting algorithm. In addition, we show linear convergence for the proximal alternating linearized minimization algorithm under a group of equivalent EB conditions, which are strictly weaker than the traditional strongly convex condition. Moreover, by defining a new EB condition, we show Q-linear convergence of Nesterov’s accelerated forward–backward algorithm without strong convexity. Finally, we verify EB conditions for a class of dual objective functions.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-018-01360-1