A uniform momentum-based distributed stochastic gradient tracking algorithm for non-convex optimization

This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applica...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the Franklin Institute Vol. 362; no. 15; p. 108040
Main Authors: Li, Yantao, Wu, Chaoxu, Chen, Yingjue, Zhang, Keke, Lü, Qingguo, Deng, Shaojiang, Li, Huaqing
Format: Journal Article
Language:English
Published: Elsevier Inc 01.10.2025
Subjects:
Distributed optimization
Gradient tracking
Non-convex optimization
Stochastic gradient
Uniform momentum acceleration
Uniform momentum acceleration
Gradient tracking
Distributed optimization
Non-convex optimization
Stochastic gradient
ISSN:0016-0032
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applications in artificial intelligence and engineering, mostly evident in machine learning, resource management, etc. To this end, we propose a novel distributed stochastic momentum acceleration algorithm which providing a unified momentum acceleration paradigm for distributed stochastic gradient tracking methods. By adjusting the parameters of the proposed algorithm, different distributed momentum acceleration methods can be obtained. In theoretical analysis, we prove that the proposed algorithm is capable of converging to a neighbourhood of a first-order stationary point of the non-convex function with a sub-linear convergence rate. Moreover, the proposed algorithm is proved to achieve convergence independent of network topology under certain conditions. Finally, numerical simulations are conducted to demonstrate the effectiveness of the proposed algorithm.
ISSN:0016-0032
DOI:10.1016/j.jfranklin.2025.108040