A variance‐reduced distributed stochastic momentum algorithm over directed networks
The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named D...
Uloženo v:
| Vydáno v: | Asian journal of control |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
02.07.2025
|
| ISSN: | 1561-8625, 1934-6093 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named DSM‐SAGA, is developed. DSM‐SAGA is a combination of the heavy‐ball momentum method and the stochastic average gradient acceleration (SAGA) method. The use of the variance reduction technique of SAGA can eliminate the variance introduced by the stochastic gradients. Furthermore, we apply the push‐sum consensus strategy to address the imbalance caused by directed networks. Under appropriate assumptions, DSM‐SAGA is proved to be linearly convergent to the optimal solution. Numerical results on a distributed logistic regression problem show that DSM‐SAGA has faster convergence than several existing distributed algorithms. |
|---|---|
| ISSN: | 1561-8625 1934-6093 |
| DOI: | 10.1002/asjc.3775 |