Distributed Stochastic Proximal Algorithm With Random Reshuffling for Nonsmooth Finite-Sum Optimization

The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differ...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems Vol. 35; no. 3; pp. 1 - 15
Main Authors: Jiang, Xia, Zeng, Xianlin, Sun, Jian, Chen, Jie, Xie, Lihua
Format: Journal Article
Language:English
Published: United States IEEE 01.03.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Convergence
Convex functions
Costs
Distributed optimization
Machine learning
Machine learning algorithms
Minimization
Multiagent systems
Objective function
Optimization
proximal operator
random reshuffling (RR)
Regularization
stochastic algorithm
Stochasticity
Sums
time-varying graphs
ISSN:2162-237X, 2162-2388, 2162-2388
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The nonsmooth finite-sum minimization is a fundamental problem in machine learning. This article develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multiagent networks. The objective function is a sum of differentiable convex functions and nonsmooth regularization. Each agent in the network updates local variables by local information exchange and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution with an <inline-formula> <tex-math notation="LaTeX">\mathcal{O}(({1}/{T})+({1}/{\sqrt{T}}))</tex-math> </inline-formula> convergence rate, where <inline-formula> <tex-math notation="LaTeX">T</tex-math> </inline-formula> is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2022.3201711