Stochastic Strongly Convex Optimization via Distributed Epoch Stochastic Gradient Algorithm

This article considers the problem of stochastic strongly convex optimization over a network of multiple interacting nodes. The optimization is under a global inequality constraint and the restriction that nodes have only access to the stochastic gradients of their objective functions. We propose an...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transaction on neural networks and learning systems Jg. 32; H. 6; S. 2344 - 2357
Hauptverfasser: Yuan, Deming, Ho, Daniel W. C., Xu, Shengyuan
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 01.06.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Approximation algorithms
Computational geometry
Convergence
Convergence rate
Convex analysis
Convex functions
Convexity
Distributed algorithms
distributed stochastic strongly optimization
epoch gradient descent
inequality constraint
Linear programming
multiagent systems
Nodes
Optimization
Probability theory
Stochastic processes
Stochasticity
ISSN:2162-237X, 2162-2388, 2162-2388
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This article considers the problem of stochastic strongly convex optimization over a network of multiple interacting nodes. The optimization is under a global inequality constraint and the restriction that nodes have only access to the stochastic gradients of their objective functions. We propose an efficient distributed non-primal-dual algorithm, by incorporating the inequality constraint into the objective via a smoothing technique. We show that the proposed algorithm achieves an optimal <inline-formula> <tex-math notation="LaTeX">\mathcal {O}((1)/(T)) </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">T </tex-math></inline-formula> is the total number of iterations) convergence rate in the mean square distance from the optimal solution. In particular, we establish a high probability bound for the proposed algorithm, by showing that with a probability at least <inline-formula> <tex-math notation="LaTeX">1-\delta </tex-math></inline-formula>, the proposed algorithm converges at a rate of <inline-formula> <tex-math notation="LaTeX">\mathcal {O}(\ln (\ln (T)/\delta)/ T) </tex-math></inline-formula>. Finally, we provide numerical experiments to demonstrate the efficacy of the proposed algorithm.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2020.3004723