A Decentralized Stochastic Algorithm for Coupled Composite Optimization With Linear Convergence

In this article, we consider a multi-node sharing problem, where each node possesses a local smooth function that is further considered as the average of several constituent functions, and the network aims to minimize a finite-sum of all local functions plus a coupling function (possibly non-smooth)...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on signal and information processing over networks Jg. 8; S. 627 - 640
Hauptverfasser: Lu, Qingguo, Liao, Xiaofeng, Deng, Shaojiang, Li, Huaqing
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Convergence
Cost function
Costs
Coupling
Couplings
Decentralized optimization
Engineering research
Information processing
linear convergence
non-smooth coupling function
Optimization
proximal primal-dual method
Robustness
Robustness (mathematics)
Saddle points
Scalability
variance-reduction
ISSN:2373-776X, 2373-7778
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this article, we consider a multi-node sharing problem, where each node possesses a local smooth function that is further considered as the average of several constituent functions, and the network aims to minimize a finite-sum of all local functions plus a coupling function (possibly non-smooth). Decentralized optimization to solve this problem has been a significant focus within engineering research due to its advantages in scalability, robustness, and flexibility. To this aim, an equivalent saddle-point problem of this problem is first formulated, which is amenable to decentralized solutions. Then, a novel decentralized stochastic algorithm, named VR-DPPD, is proposed, which combines the variance-reduction technique of SAGA with the decentralized proximal primal-dual method. We provide a convergence analysis and show that VR-DPPD converges linearly to the exact optimal solution in expectation if smooth local functions are strongly convex. Our work makes progress towards resolving a general composite optimization problem with a convex (possibly non-smooth) coupling function, giving a novel linear convergent algorithm for achieving low computation cost. Numerical examples are presented to demonstrate the viability and performance of VR-DPPD.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2373-776X
2373-7778
DOI:10.1109/TSIPN.2022.3190743