Gradient Norm Regularization Second-Order Algorithms for Solving Nonconvex-Strongly Concave Minimax Problems

In this paper, we study second-order algorithms for solving nonconvex-strongly concave minimax problems, which have attracted much attention in recent years in many fields, especially in machine learning. We propose a gradient norm regularized trust-region (GRTR) algorithm to solve nonconvex-strongl...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of scientific computing Ročník 105; číslo 2; s. 41
Hlavní autoři: Wang, Junlin, Xu, Zi
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.11.2025
Springer Nature B.V
Témata:
Algorithms
Complexity
Computational Mathematics and Numerical Analysis
Hessian matrices
Jacobi matrix method
Jacobian matrix
Machine learning
Mathematical and Computational Engineering
Mathematical and Computational Physics
Mathematics
Mathematics and Statistics
Minimax technique
Optimization algorithms
Regularization
Theoretical
Trust-region
90C30
90C47
Second-order algorithms
90C26
Levenberg-Marquardt algorithm
Nonconvex-strongly concave minimax problems
ISSN:0885-7474, 1573-7691
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we study second-order algorithms for solving nonconvex-strongly concave minimax problems, which have attracted much attention in recent years in many fields, especially in machine learning. We propose a gradient norm regularized trust-region (GRTR) algorithm to solve nonconvex-strongly concave minimax problems, where the objective function of the trust-region subproblem in each iteration uses a regularized version of the Hessian matrix, and the regularization coefficient and the radius of the ball constraint are proportional to the square root of the gradient norm. The iteration complexity of the proposed GRTR algorithm to obtain an O ( ϵ , ϵ ) -second-order stationary point is proved to be upper bounded by O ~ ( ℓ 1.5 ρ 0.5 μ - 1.5 ϵ - 1.5 ) , where μ is the strongly concave coefficient, ℓ and ρ are the Lipschitz constants of the gradient and Jacobian matrix respectively, which matches the best known iteration complexity of second-order methods for solving nonconvex-strongly concave minimax problems. We further propose a Levenberg-Marquardt algorithm with a gradient norm regularization coefficient and use the negative curvature direction to correct the iteration direction (LMNegCur), which does not need to solve the trust-region subproblem at each iteration. We also prove that the LMNegCur algorithm achieves an O ( ϵ , ϵ ) -second-order stationary point within O ~ ( ℓ 1.5 ρ 0.5 μ - 1.5 ϵ - 1.5 ) number of iterations. Furthermore, we propose two inexact variants of the above two algorithms, namely the IGRTR algorithm and the ILMNegCur algorithm, which allow to approximately solve the subproblems and still obtain O ( ϵ , ϵ ) -second-order stationary points with high probability, but only require O ~ ( ℓ 2.25 ρ 0.25 μ - 1.75 ϵ - 1.75 ) Hessian-vector products and O ~ ( ℓ 2 ρ 0.5 μ - 2 ϵ - 1.5 ) gradient ascent steps. Numerical results show the efficiency of the proposed algorithms.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7474
1573-7691
DOI:10.1007/s10915-025-03069-8