A J-symmetric quasi-newton method for minimax problems

Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric st...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Mathematical programming Ročník 204; číslo 1-2; s. 207 - 254
Hlavní autori: Asl, Azam, Lu, Haihao, Yang, Jinwen
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2024
Springer
Predmet:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Full Length Paper
Machine learning
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Methods
Numerical Analysis
Theoretical
65K15
Quasi-Newton method
Trust region
90C47
Minimax optimization
Superlinear convergence
symmetric
49K35
ISSN:0025-5610, 1436-4646
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric structure of the second-order derivative of the objective function in minimax problem. We show that the Hessian estimation (as well as its inverse) can be updated by a rank-2 operation, and it turns out that the update rule is a natural generalization of the classic Powell symmetric Broyden method from minimization problems to minimax problems. In theory, we show that our proposed quasi-Newton algorithm enjoys local Q-superlinear convergence to a desirable solution under standard regularity conditions. Furthermore, we introduce a trust-region variant of the algorithm that enjoys global R-superlinear convergence. Finally, we present numerical experiments that verify our theory and show the effectiveness of our proposed algorithms compared to Broyden’s method and the extragradient method on three classes of minimax problems.
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-023-01957-1