Stochastic Strongly Convex Optimization via Distributed Epoch Stochastic Gradient Algorithm

This article considers the problem of stochastic strongly convex optimization over a network of multiple interacting nodes. The optimization is under a global inequality constraint and the restriction that nodes have only access to the stochastic gradients of their objective functions. We propose an...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transaction on neural networks and learning systems Ročník 32; číslo 6; s. 2344 - 2357
Hlavní autori: Yuan, Deming, Ho, Daniel W. C., Xu, Shengyuan
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Piscataway IEEE 01.06.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Algorithms
Approximation algorithms
Computational geometry
Convergence
Convergence rate
Convex analysis
Convex functions
Convexity
Distributed algorithms
distributed stochastic strongly optimization
epoch gradient descent
inequality constraint
Linear programming
multiagent systems
Nodes
Optimization
Probability theory
Stochastic processes
Stochasticity
ISSN:2162-237X, 2162-2388, 2162-2388
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This article considers the problem of stochastic strongly convex optimization over a network of multiple interacting nodes. The optimization is under a global inequality constraint and the restriction that nodes have only access to the stochastic gradients of their objective functions. We propose an efficient distributed non-primal-dual algorithm, by incorporating the inequality constraint into the objective via a smoothing technique. We show that the proposed algorithm achieves an optimal <inline-formula> <tex-math notation="LaTeX">\mathcal {O}((1)/(T)) </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">T </tex-math></inline-formula> is the total number of iterations) convergence rate in the mean square distance from the optimal solution. In particular, we establish a high probability bound for the proposed algorithm, by showing that with a probability at least <inline-formula> <tex-math notation="LaTeX">1-\delta </tex-math></inline-formula>, the proposed algorithm converges at a rate of <inline-formula> <tex-math notation="LaTeX">\mathcal {O}(\ln (\ln (T)/\delta)/ T) </tex-math></inline-formula>. Finally, we provide numerical experiments to demonstrate the efficacy of the proposed algorithm.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2020.3004723