PMGT-VR: a Decentralized Proximal-gradient Algorithmic Framework with Variance Reduction

This paper considers the decentralized composite optimization problem. We propose a novel decentralized variance-reduction proximal-gradient algorithmic framework, called PMGT-VR, which combines several techniques, including multi-consensus, gradient tracking, and variance reduction. The proposed fr...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence Vol. 48; no. 1; pp. 1 - 12
Main Authors: Ye, Haishan, Xiong, Wei, Zhang, Tong
Format: Journal Article
Language:English
Published: United States IEEE 05.09.2025
Subjects:
Complexity theory
Convergence
Decentralized optimization
Loss measurement
Network topology
Optimization
proximal-gradient
Servers
Symmetric matrices
Topology
Training
variance reduction
Vectors
ISSN:0162-8828, 1939-3539, 2160-9292, 1939-3539
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper considers the decentralized composite optimization problem. We propose a novel decentralized variance-reduction proximal-gradient algorithmic framework, called PMGT-VR, which combines several techniques, including multi-consensus, gradient tracking, and variance reduction. The proposed framework imitates centralized algorithms and algorithms under this framework achieve convergence rates similar to that of their centralized counterparts. We also describe and analyze two representative algorithms, PMGT-SAGA and PMGT-LSVRG, and compare them to existing state-of-the-art proximal algorithms. To the best of our knowledge, PMGT-VR is the first linearly convergent decentralized stochastic algorithm that can solve decentralized composite optimization problems. Numerical experiments are provided to demonstrate the effectiveness of the proposed algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0162-8828
1939-3539
2160-9292
1939-3539
DOI:10.1109/TPAMI.2025.3606874