A J-symmetric quasi-newton method for minimax problems

Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric st...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Mathematical programming Jg. 204; H. 1-2; S. 207 - 254
Hauptverfasser: Asl, Azam, Lu, Haihao, Yang, Jinwen
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2024
Springer
Schlagworte:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Full Length Paper
Machine learning
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Methods
Numerical Analysis
Theoretical
65K15
Quasi-Newton method
Trust region
90C47
Minimax optimization
Superlinear convergence
symmetric
49K35
ISSN:0025-5610, 1436-4646
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric structure of the second-order derivative of the objective function in minimax problem. We show that the Hessian estimation (as well as its inverse) can be updated by a rank-2 operation, and it turns out that the update rule is a natural generalization of the classic Powell symmetric Broyden method from minimization problems to minimax problems. In theory, we show that our proposed quasi-Newton algorithm enjoys local Q-superlinear convergence to a desirable solution under standard regularity conditions. Furthermore, we introduce a trust-region variant of the algorithm that enjoys global R-superlinear convergence. Finally, we present numerical experiments that verify our theory and show the effectiveness of our proposed algorithms compared to Broyden’s method and the extragradient method on three classes of minimax problems.
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-023-01957-1