Implementable tensor methods in unconstrained convex optimization

In this paper we develop new tensor methods for unconstrained convex optimization, which solve at each iteration an auxiliary problem of minimizing convex multivariate polynomial. We analyze the simplest scheme, based on minimization of a regularized local model of the objective function, and its ac...

Full description

Saved in:
Bibliographic Details
Published in:Mathematical programming Vol. 186; no. 1-2; pp. 157 - 183
Main Author: Nesterov, Yurii
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2021
Springer Nature B.V
Subjects:
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Computational geometry
Convergence
Convex analysis
Convexity
Full Length Paper
Iterative methods
Lower bounds
Mathematical analysis
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Numerical Analysis
Optimization
Polynomials
Sequences
Smoothness
Tensors
Theoretical
65K05
High-order methods
Convex optimization
90C25
Lower complexity bounds
90C06
Worst-case complexity bounds
Tensor methods
ISSN:0025-5610, 1436-4646
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper we develop new tensor methods for unconstrained convex optimization, which solve at each iteration an auxiliary problem of minimizing convex multivariate polynomial. We analyze the simplest scheme, based on minimization of a regularized local model of the objective function, and its accelerated version obtained in the framework of estimating sequences. Their rates of convergence are compared with the worst-case lower complexity bounds for corresponding problem classes. Finally, for the third-order methods, we suggest an efficient technique for solving the auxiliary problem, which is based on the recently developed relative smoothness condition (Bauschke et al. in Math Oper Res 42:330–348, 2017; Lu et al. in SIOPT 28(1):333–354, 2018). With this elaboration, the third-order methods become implementable and very fast. The rate of convergence in terms of the function value for the accelerated third-order scheme reaches the level O 1 k 4 , where k is the number of iterations. This is very close to the lower bound of the order O 1 k 5 , which is also justified in this paper. At the same time, in many important cases the computational cost of one iteration of this method remains on the level typical for the second-order methods.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-019-01449-1