Superfast Second-Order Methods for Unconstrained Convex Optimization

In this paper, we present new second-order methods with convergence rate O k - 4 , where k is the iteration counter. This is faster than the existing lower bound for this type of schemes (Agarwal and Hazan in Proceedings of the 31st conference on learning theory, PMLR, pp. 774–792, 2018; Arjevani an...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of optimization theory and applications Ročník 191; číslo 1; s. 1 - 30
Hlavní autor: Nesterov, Yurii
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.10.2021
Springer Nature B.V
Springer Verlag
Témata:
Accuracy
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Computational geometry
Convergence
Convex analysis
Convexity
Engineering
Iterative methods
Learning theory
Lower bounds
Mathematics
Mathematics and Statistics
Methods
Neighborhoods
Operations Research/Decision Theory
Optimization
Optimization and Control
Theory of Computation
Convex optimization
90C25
Lower complexity bounds
Second-order methods
Tensor methods
ISSN:0022-3239, 1573-2878
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we present new second-order methods with convergence rate O k - 4 , where k is the iteration counter. This is faster than the existing lower bound for this type of schemes (Agarwal and Hazan in Proceedings of the 31st conference on learning theory, PMLR, pp. 774–792, 2018; Arjevani and Shiff in Math Program 178(1–2):327–360, 2019), which is O k - 7 / 2 . Our progress can be explained by a finer specification of the problem class. The main idea of this approach consists in implementation of the third-order scheme from Nesterov (Math Program 186:157–183, 2021) using the second-order oracle. At each iteration of our method, we solve a nontrivial auxiliary problem by a linearly convergent scheme based on the relative non-degeneracy condition (Bauschke et al. in Math Oper Res 42:330–348, 2016; Lu et al. in SIOPT 28(1):333–354, 2018). During this process, the Hessian of the objective function is computed once, and the gradient is computed O ln 1 ϵ times, where ϵ is the desired accuracy of the solution for our problem.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-021-01930-y