On the convergence and improvement of stochastic normalized gradient descent

Non-convex models, like deep neural networks, have been widely used in machine learning applications. Training non-convex models is a difficult task owing to the saddle points of models. Recently, stochastic normalized gradient descent (SNGD), which updates the model parameter by a normalized gradie...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	Science China. Information sciences Jg. 64; H. 3; S. 132103
Hauptverfasser:	Zhao, Shen-Yi, Xie, Yin-Peng, Li, Wu-Jun
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Beijing Science China Press 01.03.2021 Springer Nature B.V
Schlagworte:	Artificial neural networks Computer Science Convergence Information Systems and Communication Service Iterative methods Machine learning Neural networks Performance enhancement Research Paper Saddle points stochastic normalized gradient descent non-convex problems computation complexity
ISSN:	1674-733X, 1869-1919
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!