A variance‐reduced distributed stochastic momentum algorithm over directed networks
The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named D...
Uložené v:
| Vydané v: | Asian journal of control |
|---|---|
| Hlavní autori: | , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
02.07.2025
|
| ISSN: | 1561-8625, 1934-6093 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Shrnutí: | The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named DSM‐SAGA, is developed. DSM‐SAGA is a combination of the heavy‐ball momentum method and the stochastic average gradient acceleration (SAGA) method. The use of the variance reduction technique of SAGA can eliminate the variance introduced by the stochastic gradients. Furthermore, we apply the push‐sum consensus strategy to address the imbalance caused by directed networks. Under appropriate assumptions, DSM‐SAGA is proved to be linearly convergent to the optimal solution. Numerical results on a distributed logistic regression problem show that DSM‐SAGA has faster convergence than several existing distributed algorithms. |
|---|---|
| ISSN: | 1561-8625 1934-6093 |
| DOI: | 10.1002/asjc.3775 |