A uniform momentum-based distributed stochastic gradient tracking algorithm for non-convex optimization

This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applica...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of the Franklin Institute Jg. 362; H. 15; S. 108040
Hauptverfasser: Li, Yantao, Wu, Chaoxu, Chen, Yingjue, Zhang, Keke, Lü, Qingguo, Deng, Shaojiang, Li, Huaqing
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Inc 01.10.2025
Schlagworte:
Distributed optimization
Gradient tracking
Non-convex optimization
Stochastic gradient
Uniform momentum acceleration
Uniform momentum acceleration
Gradient tracking
Distributed optimization
Non-convex optimization
Stochastic gradient
ISSN:0016-0032
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper investigates distributed non-convex optimization problems, specifically focusing on a collaborative approach to optimize a global non-convex objective function across agents over networks. Problems of this nature along with consideration of efficiency usually emerge in a number of applications in artificial intelligence and engineering, mostly evident in machine learning, resource management, etc. To this end, we propose a novel distributed stochastic momentum acceleration algorithm which providing a unified momentum acceleration paradigm for distributed stochastic gradient tracking methods. By adjusting the parameters of the proposed algorithm, different distributed momentum acceleration methods can be obtained. In theoretical analysis, we prove that the proposed algorithm is capable of converging to a neighbourhood of a first-order stationary point of the non-convex function with a sub-linear convergence rate. Moreover, the proposed algorithm is proved to achieve convergence independent of network topology under certain conditions. Finally, numerical simulations are conducted to demonstrate the effectiveness of the proposed algorithm.
ISSN:0016-0032
DOI:10.1016/j.jfranklin.2025.108040