A Randomized Algorithm for Nonconvex Minimization With Inexact Evaluations and Complexity Guarantees

We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of optimization theory and applications Ročník 207; číslo 3; s. 66
Hlavní autoři: Li, Shuyao, Wright, Stephen J.
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.12.2025
Springer Nature B.V
Témata:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Complexity
Engineering
Folklore
Martingales
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Probability
Regularization methods
Theory of Computation
68W20
Nonconvex optimization
Second-order methods
90C17
90C26
Second-order stationarity
Inexact Hessian
Inexact gradient
ISSN:0022-3239, 1573-2878
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sign to be positive or negative with equal probability. We allow gradients to be inexact (with a bound on their error relative to the size of the true quantity) and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-025-02817-y