A trust region algorithm with a worst-case iteration complexity of O(ϵ-3/2) for nonconvex optimization

We propose a trust region algorithm for solving nonconvex smooth optimization problems. For any ϵ ¯ ∈ ( 0 , ∞ ) , the algorithm requires at most O ( ϵ - 3 / 2 ) iterations, function evaluations, and derivative evaluations to drive the norm of the gradient of the objective function below any ϵ ∈ ( 0...

Full description

Saved in:
Bibliographic Details
Published in:Mathematical programming Vol. 162; no. 1-2; pp. 1 - 32
Main Authors: Curtis, Frank E., Robinson, Daniel P., Samadi, Mohammadreza
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2017
Springer Nature B.V
Subjects:
Applied mathematics
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Full Length Paper
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Numerical Analysis
Optimization algorithms
Regularization methods
Theoretical
58C15
68Q25
Global convergence
Worst-case iteration complexity
Unconstrained optimization
Nonconvex optimization
65K05
Nonlinear optimization
65Y20
90C30
65K10
49M15
49M37
Local convergence
90C60
Trust region methods
Worst-case evaluation complexity
ISSN:0025-5610, 1436-4646
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a trust region algorithm for solving nonconvex smooth optimization problems. For any ϵ ¯ ∈ ( 0 , ∞ ) , the algorithm requires at most O ( ϵ - 3 / 2 ) iterations, function evaluations, and derivative evaluations to drive the norm of the gradient of the objective function below any ϵ ∈ ( 0 , ϵ ¯ ] . This improves upon the O ( ϵ - 2 ) bound known to hold for some other trust region algorithms and matches the O ( ϵ - 3 / 2 ) bound for the recently proposed Adaptive Regularisation framework using Cubics, also known as the arc algorithm. Our algorithm, entitled trace , follows a trust region framework, but employs modified step acceptance criteria and a novel trust region update mechanism that allow the algorithm to achieve such a worst-case global complexity bound. Importantly, we prove that our algorithm also attains global and fast local convergence guarantees under similar assumptions as for other trust region algorithms. We also prove a worst-case upper bound on the number of iterations, function evaluations, and derivative evaluations that the algorithm requires to obtain an approximate second-order stationary point.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-016-1026-2