SABRINA: A Stochastic Subspace Majorization-Minimization Algorithm

A wide class of problems involves the minimization of a coercive and differentiable function F on R N whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustne...

Full description

Saved in:
Bibliographic Details
Published in:Journal of optimization theory and applications Vol. 195; no. 3; pp. 919 - 952
Main Authors: Chouzenoux, Emilie, Fest, Jean-Baptiste
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2022
Springer Nature B.V
Springer Verlag
Subjects:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Convergence
Engineering
Engineering Sciences
Errors
Image processing
Machine learning
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Robustness (mathematics)
Signal and Image processing
Stochastic processes
Subspaces
Theory of Computation
Subspace acceleration
Majorization-minimization
Stochastic optimization
Binary logistic regression
Image reconstruction
Convergence analysis
subspace acceleration
binary logistic regression
image reconstruction
convergence analysis
Majorization-Minimization
ISSN:0022-3239, 1573-2878
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A wide class of problems involves the minimization of a coercive and differentiable function F on R N whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustness to errors in the gradient is not necessarily guaranteed. This work is dedicated to investigating the convergence of Majorization-Minimization (MM) schemes when stochastic errors affect the gradient terms. We introduce a general stochastic optimization framework, called StochAstic suBspace majoRIzation-miNimization Algorithm SABRINA that encompasses MM quadratic schemes possibly enhanced with a subspace acceleration strategy. New asymptotical results are built for the stochastic process generated by SABRINA . Two sets of numerical experiments in the field of machine learning and image processing are presented to support our theoretical results and illustrate the good performance of SABRINA with respect to state-of-the-art gradient-based stochastic optimization methods.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-022-02122-y