Riemannian Stochastic Variance-Reduced Cubic Regularized Newton Method for Submanifold Optimization

We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the beginning of each epoch while it performs stochastic variance...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of optimization theory and applications Jg. 196; H. 1; S. 324 - 361
Hauptverfasser: Zhang, Dewei, Davanloo Tajbakhsh, Sam
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.01.2023
Springer Nature B.V
Schlagworte:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Complexity
Eigenvalues
Engineering
Euclidean geometry
Euclidean space
Iterative methods
Manifolds (mathematics)
Mathematics
Mathematics and Statistics
Matrices (mathematics)
Newton methods
Numerical methods
Operations Research/Decision Theory
Optimization
Theory of Computation
Stochastic optimization
Cubic regularization
Manifold optimization
Variance reduction
Riemannian optimization
ISSN:0022-3239, 1573-2878
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the beginning of each epoch while it performs stochastic variance-reduced updates in the iterations within each epoch. The iteration complexity of O ( ϵ - 3 / 2 ) to obtain an ( ϵ , ϵ ) -second-order stationary point, i.e., a point with the Riemannian gradient norm upper bounded by ϵ and minimum eigenvalue of Riemannian Hessian lower bounded by - ϵ , is established when the manifold is embedded in the Euclidean space. Furthermore, the paper proposes a computationally more appealing modification of the algorithm which only requires an inexact solution of the cubic regularized Newton subproblem with the same iteration complexity. The proposed algorithm is evaluated and compared with three other Riemannian second-order methods over two numerical studies on estimating the inverse scale matrix of the multivariate t-distribution on the manifold of symmetric positive definite matrices and estimating the parameter of a linear classifier on the sphere manifold.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-022-02137-5