Distributed Stochastic Proximal Algorithm With Random Reshuffling for Nonsmooth Finite-Sum Optimization

The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differ...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transaction on neural networks and learning systems Ročník 35; číslo 3; s. 1 - 15
Hlavní autori: Jiang, Xia, Zeng, Xianlin, Sun, Jian, Chen, Jie, Xie, Lihua
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: United States IEEE 01.03.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Algorithms
Convergence
Convex functions
Costs
Distributed optimization
Machine learning
Machine learning algorithms
Minimization
Multiagent systems
Objective function
Optimization
proximal operator
random reshuffling (RR)
Regularization
stochastic algorithm
Stochasticity
Sums
time-varying graphs
ISSN:2162-237X, 2162-2388, 2162-2388
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differentiable convex functions and nonsmooth regularization. Each agent in the network updates local variables by local information exchange and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution with an <inline-formula> <tex-math notation="LaTeX">\mathcal{O}(({1}/{T})+({1}/{\sqrt{T}}))</tex-math> </inline-formula> convergence rate, where <inline-formula> <tex-math notation="LaTeX">T</tex-math> </inline-formula> is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2022.3201711