SABRINA: A Stochastic Subspace Majorization-Minimization Algorithm

A wide class of problems involves the minimization of a coercive and differentiable function F on R N whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustne...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of optimization theory and applications Jg. 195; H. 3; S. 919 - 952
Hauptverfasser: Chouzenoux, Emilie, Fest, Jean-Baptiste
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.12.2022
Springer Nature B.V
Springer Verlag
Schlagworte:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Convergence
Engineering
Engineering Sciences
Errors
Image processing
Machine learning
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Robustness (mathematics)
Signal and Image processing
Stochastic processes
Subspaces
Theory of Computation
Subspace acceleration
Majorization-minimization
Stochastic optimization
Binary logistic regression
Image reconstruction
Convergence analysis
subspace acceleration
binary logistic regression
image reconstruction
convergence analysis
Majorization-Minimization
ISSN:0022-3239, 1573-2878
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A wide class of problems involves the minimization of a coercive and differentiable function F on R N whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustness to errors in the gradient is not necessarily guaranteed. This work is dedicated to investigating the convergence of Majorization-Minimization (MM) schemes when stochastic errors affect the gradient terms. We introduce a general stochastic optimization framework, called StochAstic suBspace majoRIzation-miNimization Algorithm SABRINA that encompasses MM quadratic schemes possibly enhanced with a subspace acceleration strategy. New asymptotical results are built for the stochastic process generated by SABRINA . Two sets of numerical experiments in the field of machine learning and image processing are presented to support our theoretical results and illustrate the good performance of SABRINA with respect to state-of-the-art gradient-based stochastic optimization methods.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-022-02122-y