A trust region algorithm with a worst-case iteration complexity of O(ϵ-3/2) for nonconvex optimization - Curtis, Frank E.; Robinson, Daniel P.; Samadi, Mohammadreza (Mathematical programming, 2017)
We propose a trust region algorithm for solving nonconvex smooth optimization problems. For any ϵ ¯ ∈ ( 0 , ∞ ) , the algorithm requires at most O ( ϵ - 3 / 2 ) iterations, function evaluations, and derivative evaluations to drive the norm of the gradient of the objective function below any ϵ ∈ ( 0 , ϵ ¯ ] . This improves upon the O ( ϵ - 2 ) bound known to hold for some other trust region algorithms and matches the O ( ϵ - 3 / 2 ) bound for the recently proposed Adaptive Regularisation framework using Cubics, also known as the arc algorithm. Our algorithm, entitled trace , follows a trust region framework, but employs modified step acceptance criteria and a novel trust region update mechanism that allow the algorithm to achieve such a worst-case global complexity bound. Importantly, we prove that our algorithm also attains global and fast local convergence guarantees under similar assumptions as for other trust region algorithms. We also prove a worst-case upper bound on the number of iterations, function evaluations, and derivative evaluations that the algorithm requires to obtain an approximate second-order stationary point.