Hybrid SGD algorithms to solve stochastic composite optimization problems with application in sparse portfolio selection problems

In this paper, we study stochastic composite problems where the objective can be the composition of an outer single-valued function and an inner vector-valued mapping. In this stochastic composite optimization, the inner mapping can be expressed as an expectation over random component mappings. In t...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of computational and applied mathematics Jg. 436; S. 115425
Hauptverfasser: Yang, Zhen-Ping, Zhao, Yong
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier B.V 15.01.2024
Schlagworte:
Complexity
Hybrid stochastic estimator
Normalized proximal gradient algorithm
Prox-linear algorithm
Stochastic nonsmooth composite optimization
68W20
68Q25
Stochastic nonsmooth composite optimization
90C26
Hybrid stochastic estimator
Normalized proximal gradient algorithm
Prox-linear algorithm
Complexity
ISSN:0377-0427, 1879-1778
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, we study stochastic composite problems where the objective can be the composition of an outer single-valued function and an inner vector-valued mapping. In this stochastic composite optimization, the inner mapping can be expressed as an expectation over random component mappings. In this study, we propose two algorithms to address the generality and possible singularities of this problem and bound their sample complexities for finding an ϵ-stationary point. The first algorithm is the prox-linear hybrid stochastic gradient algorithm, which may achieve sample complexities of O(ϵ2τ−5/2) and O(ϵτ−3/2) for the component mappings and their Jacobians respectively, where τ∈[0,1]. The second algorithm is the normalized proximal hybrid stochastic gradient algorithm, which takes advantage of the special structure of the regularizer. This algorithm may achieve sample complexities of O(ϵ2τ−4) for both the component mappings and the Jacobians, where τ∈[5/4,7/4]. Numerical experiments prove that the two proposed algorithms are quite competitive with other existing algorithms. A real-life application in sparse portfolio selection problems is also promising. •We study a class of stochastic composite optimization problems.•We present two hybrid SGD algorithms for the considered problem.•We investigate the convergence rates and complexity of the algorithms.•We apply the proposed approach to the sparse portfolio selection problem.
ISSN:0377-0427
1879-1778
DOI:10.1016/j.cam.2023.115425