Hybrid SGD algorithms to solve stochastic composite optimization problems with application in sparse portfolio selection problems

In this paper, we study stochastic composite problems where the objective can be the composition of an outer single-valued function and an inner vector-valued mapping. In this stochastic composite optimization, the inner mapping can be expressed as an expectation over random component mappings. In t...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Journal of computational and applied mathematics Ročník 436; s. 115425
Hlavní autori: Yang, Zhen-Ping, Zhao, Yong
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier B.V 15.01.2024
Predmet:
Complexity
Hybrid stochastic estimator
Normalized proximal gradient algorithm
Prox-linear algorithm
Stochastic nonsmooth composite optimization
68W20
68Q25
Stochastic nonsmooth composite optimization
90C26
Hybrid stochastic estimator
Normalized proximal gradient algorithm
Prox-linear algorithm
Complexity
ISSN:0377-0427, 1879-1778
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In this paper, we study stochastic composite problems where the objective can be the composition of an outer single-valued function and an inner vector-valued mapping. In this stochastic composite optimization, the inner mapping can be expressed as an expectation over random component mappings. In this study, we propose two algorithms to address the generality and possible singularities of this problem and bound their sample complexities for finding an ϵ-stationary point. The first algorithm is the prox-linear hybrid stochastic gradient algorithm, which may achieve sample complexities of O(ϵ2τ−5/2) and O(ϵτ−3/2) for the component mappings and their Jacobians respectively, where τ∈[0,1]. The second algorithm is the normalized proximal hybrid stochastic gradient algorithm, which takes advantage of the special structure of the regularizer. This algorithm may achieve sample complexities of O(ϵ2τ−4) for both the component mappings and the Jacobians, where τ∈[5/4,7/4]. Numerical experiments prove that the two proposed algorithms are quite competitive with other existing algorithms. A real-life application in sparse portfolio selection problems is also promising. •We study a class of stochastic composite optimization problems.•We present two hybrid SGD algorithms for the considered problem.•We investigate the convergence rates and complexity of the algorithms.•We apply the proposed approach to the sparse portfolio selection problem.
ISSN:0377-0427
1879-1778
DOI:10.1016/j.cam.2023.115425