Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results

An Adaptive Regularisation algorithm using Cubics (ARC) is proposed for unconstrained optimization, generalizing at the same time an unpublished method due to Griewank (Technical Report NA/12, 1981, DAMTP, University of Cambridge), an algorithm by Nesterov and Polyak (Math Program 108(1):177–205, 20...

Full description

Saved in:
Bibliographic Details
Published in:Mathematical programming Vol. 127; no. 2; pp. 245 - 295
Main Authors: Cartis, Coralia, Gould, Nicholas I. M., Toint, Philippe L.
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer-Verlag 01.04.2011
Springer
Springer Nature B.V
Subjects:
Adaptive algorithms
Algorithms
Applied sciences
Approximation
Arc cutting
Calculus of variations and optimal control
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Convergence
Exact sciences and technology
Full Length Paper
Laboratories
Mathematical analysis
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematical models
Mathematical programming
Mathematics
Mathematics and Statistics
Mathematics of Computing
Methods
Nonlinear programming
Numerical Analysis
Operational research and scientific management
Operational research. Management science
Optimization
Proposals
Sciences and techniques of general use
Studies
Theoretical
58C15
Global convergence
Unconstrained optimization
65K05
Nonlinear optimization
Newton’s method
Trust-region methods
90C30
65F10
49M37
49M15
Local convergence
65H05
Cubic regularization
Lipschitz condition
Adaptive algorithm
Small scale
Iterative method
Non linear programming
Modeling
Adaptive method
Lipschitz function
Newton method
Numerical convergence
Mathematical programming
Large scale system
Trust region method
Global solution
Newton's method
Objective function
Regularization
ISSN:0025-5610, 1436-4646
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An Adaptive Regularisation algorithm using Cubics (ARC) is proposed for unconstrained optimization, generalizing at the same time an unpublished method due to Griewank (Technical Report NA/12, 1981, DAMTP, University of Cambridge), an algorithm by Nesterov and Polyak (Math Program 108(1):177–205, 2006) and a proposal by Weiser et al. (Optim Methods Softw 22(3):413–431, 2007). At each iteration of our approach, an approximate global minimizer of a local cubic regularisation of the objective function is determined, and this ensures a significant improvement in the objective so long as the Hessian of the objective is locally Lipschitz continuous. The new method uses an adaptive estimation of the local Lipschitz constant and approximations to the global model-minimizer which remain computationally-viable even for large-scale problems. We show that the excellent global and local convergence properties obtained by Nesterov and Polyak are retained, and sometimes extended to a wider class of problems, by our ARC approach. Preliminary numerical experiments with small-scale test problems from the CUTEr set show encouraging performance of the ARC algorithm when compared to a basic trust-region implementation.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-2
content type line 23
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-009-0286-5