On lower iteration complexity bounds for the convex concave saddle point problems
In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: min x max y F ( x , y ) . We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proxi...
Uloženo v:
| Vydáno v: | Mathematical programming Ročník 194; číslo 1-2; s. 901 - 935 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Berlin/Heidelberg
Springer Berlin Heidelberg
01.07.2022
Springer Springer Nature B.V |
| Témata: | |
| ISSN: | 0025-5610, 1436-4646 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem:
min
x
max
y
F
(
x
,
y
)
. We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proximal mappings. For problems with gradient Lipschitz constants (
L
x
,
L
y
and
L
xy
) and strong convexity/concavity constants (
μ
x
and
μ
y
), the class of pure first-order algorithms with the linear span assumption is shown to have a lower iteration complexity bound of
Ω
L
x
μ
x
+
L
xy
2
μ
x
μ
y
+
L
y
μ
y
·
ln
1
ϵ
, where the term
L
xy
2
μ
x
μ
y
explains how the coupling influences the iteration complexity. Under several special parameter regimes, this lower bound has been achieved by corresponding optimal algorithms. However, whether or not the bound under the general parameter regime is optimal remains open. Additionally, for the special case of bilinear coupling problems, given the availability of certain proximal operators, a lower bound of
Ω
L
xy
2
μ
x
μ
y
·
ln
(
1
ϵ
)
is established under the linear span assumption, and optimal algorithms have already been developed in the literature. By exploiting the orthogonal invariance technique, we extend both lower bounds to the general pure first-order algorithm class and the proximal algorithm class without the linear span assumption. As an application, we apply proper scaling to the worst-case instances, and we derive the lower bounds for the general convex concave problems with
μ
x
=
μ
y
=
0
. Several existing results in this case can be deduced from our results as special cases. |
|---|---|
| Bibliografie: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ISSN: | 0025-5610 1436-4646 |
| DOI: | 10.1007/s10107-021-01660-z |