On lower iteration complexity bounds for the convex concave saddle point problems

In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: min x max y F ( x , y ) . We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proxi...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Mathematical programming Jg. 194; H. 1-2; S. 901 - 935
Hauptverfasser: Zhang, Junyu, Hong, Mingyi, Zhang, Shuzhong
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Berlin/Heidelberg Springer Berlin Heidelberg 01.07.2022
Springer
Springer Nature B.V
Schlagworte:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Complexity
Concavity
Convexity
Coupling
First order algorithms
Full Length Paper
Lower bounds
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Numerical Analysis
Parameters
Saddle points
Theoretical
Proximal mapping
90C47
Saddle point
First-order method
Lower iteration complexity bound
Min-max problem
ISSN:0025-5610, 1436-4646
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: min x max y F ( x , y ) . We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proximal mappings. For problems with gradient Lipschitz constants ( L x , L y and L xy ) and strong convexity/concavity constants ( μ x and μ y ), the class of pure first-order algorithms with the linear span assumption is shown to have a lower iteration complexity bound of Ω L x μ x + L xy 2 μ x μ y + L y μ y · ln 1 ϵ , where the term L xy 2 μ x μ y explains how the coupling influences the iteration complexity. Under several special parameter regimes, this lower bound has been achieved by corresponding optimal algorithms. However, whether or not the bound under the general parameter regime is optimal remains open. Additionally, for the special case of bilinear coupling problems, given the availability of certain proximal operators, a lower bound of Ω L xy 2 μ x μ y · ln ( 1 ϵ ) is established under the linear span assumption, and optimal algorithms have already been developed in the literature. By exploiting the orthogonal invariance technique, we extend both lower bounds to the general pure first-order algorithm class and the proximal algorithm class without the linear span assumption. As an application, we apply proper scaling to the worst-case instances, and we derive the lower bounds for the general convex concave problems with μ x = μ y = 0 . Several existing results in this case can be deduced from our results as special cases.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-021-01660-z