On the convergence and improvement of stochastic normalized gradient descent

Non-convex models, like deep neural networks, have been widely used in machine learning applications. Training non-convex models is a difficult task owing to the saddle points of models. Recently, stochastic normalized gradient descent (SNGD), which updates the model parameter by a normalized gradie...

Full description

Bibliographic Details

Published in:	Science China. Information sciences Vol. 64; no. 3; p. 132103
Main Authors:	Zhao, Shen-Yi, Xie, Yin-Peng, Li, Wu-Jun
Format:	Journal Article
Language:	English
Published:	Beijing Science China Press 01.03.2021 Springer Nature B.V
Subjects:	Artificial neural networks Computer Science Convergence Information Systems and Communication Service Iterative methods Machine learning Neural networks Performance enhancement Research Paper Saddle points stochastic normalized gradient descent non-convex problems computation complexity
ISSN:	1674-733X, 1869-1919
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!