A uniform momentum-based distributed stochastic gradient tracking algorithm for non-convex optimization

This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applica...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of the Franklin Institute Ročník 362; číslo 15; s. 108040
Hlavní autoři: Li, Yantao, Wu, Chaoxu, Chen, Yingjue, Zhang, Keke, Lü, Qingguo, Deng, Shaojiang, Li, Huaqing
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Inc 01.10.2025
Témata:
Distributed optimization
Gradient tracking
Non-convex optimization
Stochastic gradient
Uniform momentum acceleration
Uniform momentum acceleration
Gradient tracking
Distributed optimization
Non-convex optimization
Stochastic gradient
ISSN:0016-0032
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applications in artificial intelligence and engineering, mostly evident in machine learning, resource management, etc. To this end, we propose a novel distributed stochastic momentum acceleration algorithm which providing a unified momentum acceleration paradigm for distributed stochastic gradient tracking methods. By adjusting the parameters of the proposed algorithm, different distributed momentum acceleration methods can be obtained. In theoretical analysis, we prove that the proposed algorithm is capable of converging to a neighbourhood of a first-order stationary point of the non-convex function with a sub-linear convergence rate. Moreover, the proposed algorithm is proved to achieve convergence independent of network topology under certain conditions. Finally, numerical simulations are conducted to demonstrate the effectiveness of the proposed algorithm.
ISSN:0016-0032
DOI:10.1016/j.jfranklin.2025.108040