A J-symmetric quasi-newton method for minimax problems

Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric st...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Mathematical programming Ročník 204; číslo 1-2; s. 207 - 254
Hlavní autoři: Asl, Azam, Lu, Haihao, Yang, Jinwen
Médium: Journal Article
Jazyk:angličtina
Vydáno: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2024
Springer
Témata:
Algorithms
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Full Length Paper
Machine learning
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Methods
Numerical Analysis
Theoretical
65K15
Quasi-Newton method
Trust region
90C47
Minimax optimization
Superlinear convergence
symmetric
49K35
ISSN:0025-5610, 1436-4646
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for the minimax problems, which we call J -symmetric quasi-Newton method. The method is obtained by exploiting the J -symmetric structure of the second-order derivative of the objective function in minimax problem. We show that the Hessian estimation (as well as its inverse) can be updated by a rank-2 operation, and it turns out that the update rule is a natural generalization of the classic Powell symmetric Broyden method from minimization problems to minimax problems. In theory, we show that our proposed quasi-Newton algorithm enjoys local Q-superlinear convergence to a desirable solution under standard regularity conditions. Furthermore, we introduce a trust-region variant of the algorithm that enjoys global R-superlinear convergence. Finally, we present numerical experiments that verify our theory and show the effectiveness of our proposed algorithms compared to Broyden’s method and the extragradient method on three classes of minimax problems.
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-023-01957-1