Non-Asymptotic Analysis of Hybrid SPG for Non-Convex Stochastic Composite Optimization

This paper focuses on the stochastic composite optimization problem, wherein the objective function comprises a smooth non-convex term and a non-smooth, possibly non-convex regularizer. Existing algorithms for addressing such problems remain limited and mostly have unsatisfactory complexity. To impr...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of optimization theory and applications Ročník 207; číslo 1; s. 19
Hlavní autoři: He, Yue-Hong, Li, Gao-Xi, Long, Xian-Jun
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.10.2025
Springer Nature B.V
Témata:
Algorithms
Applications of Mathematics
Calculus of Variations and Optimal Control; Optimization
Complexity
Engineering
Euclidean space
Mathematics
Mathematics and Statistics
Operations Research/Decision Theory
Optimization
Random variables
Restarting
Theory of Computation
Non-asymptotic convergence
65K15
90C33
Hybrid stochastic estimator
90C15
Non-smooth non-convex regularizer
Stochastic proximal gradient algorithm
Complexity
ISSN:0022-3239, 1573-2878
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper focuses on the stochastic composite optimization problem, wherein the objective function comprises a smooth non-convex term and a non-smooth, possibly non-convex regularizer. Existing algorithms for addressing such problems remain limited and mostly have unsatisfactory complexity. To improve the sample complexity, we propose a hybrid stochastic proximal gradient algorithm and its restarting variant for both expectation and finite-sum problems. Our approach relies on a novel hybrid stochastic estimator that effectively balances variance and bias, avoiding unnecessary computation waste. Under mild assumptions, we prove that the proposed algorithms non-asymptotically converge to an ϵ -stationary point at a rate of O ( 1 / T ) , where T denotes the number of iterations. The sample complexity manifests as a piecewise function, which outperforms some existing state-of-the-art results. Additionally, we derive the linear convergence of the restarting algorithm based on the Kurdyka- ojasiewicz property with an exponent of 1/2. To validate the effectiveness of our algorithm, we apply them to solve large-scale linear regression and regularized loss minimization problems, demonstrating certain superiority over several existing methods.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0022-3239
1573-2878
DOI:10.1007/s10957-025-02771-9