A variance‐reduced distributed stochastic momentum algorithm over directed networks
The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named D...
Saved in:
| Published in: | Asian journal of control |
|---|---|
| Main Authors: | , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
02.07.2025
|
| ISSN: | 1561-8625, 1934-6093 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named DSM‐SAGA, is developed. DSM‐SAGA is a combination of the heavy‐ball momentum method and the stochastic average gradient acceleration (SAGA) method. The use of the variance reduction technique of SAGA can eliminate the variance introduced by the stochastic gradients. Furthermore, we apply the push‐sum consensus strategy to address the imbalance caused by directed networks. Under appropriate assumptions, DSM‐SAGA is proved to be linearly convergent to the optimal solution. Numerical results on a distributed logistic regression problem show that DSM‐SAGA has faster convergence than several existing distributed algorithms. |
|---|---|
| ISSN: | 1561-8625 1934-6093 |
| DOI: | 10.1002/asjc.3775 |