A Randomized Algorithm for Nonconvex Minimization With Inexact Evaluations and Complexity Guarantees

We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen...

Full description

Saved in:
Bibliographic Details
Published in:Journal of optimization theory and applications Vol. 207; no. 3; p. 66
Main Authors: Li, Shuyao, Wright, Stephen J.
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2025
Springer Nature B.V
Subjects:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Complexity
Engineering
Folklore
Martingales
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Probability
Regularization methods
Theory of Computation
68W20
Nonconvex optimization
Second-order methods
90C17
90C26
Second-order stationarity
Inexact Hessian
Inexact gradient
ISSN:0022-3239, 1573-2878
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sign to be positive or negative with equal probability. We allow gradients to be inexact (with a bound on their error relative to the size of the true quantity) and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-025-02817-y