A variance‐reduced distributed stochastic momentum algorithm over directed networks - Gong, Xiuhui; Gao, Juan; Liu, Xinwei (Asian journal of control, 2025)
The distributed optimization to minimize a smooth and strongly convex function is considered, in which the function can be described as the finite sum of all local objective functions over directed networks. A variance‐reduced distributed stochastic momentum algorithm over directed networks, named DSM‐SAGA, is developed. DSM‐SAGA is a combination of the heavy‐ball momentum method and the stochastic average gradient acceleration (SAGA) method. The use of the variance reduction technique of SAGA can eliminate the variance introduced by the stochastic gradients. Furthermore, we apply the push‐sum consensus strategy to address the imbalance caused by directed networks. Under appropriate assumptions, DSM‐SAGA is proved to be linearly convergent to the optimal solution. Numerical results on a distributed logistic regression problem show that DSM‐SAGA has faster convergence than several existing distributed algorithms.