Unified Algorithm Framework for Nonconvex Stochastic Optimization in Deep Neural Networks

This paper presents a unified algorithmic framework for nonconvex stochastic optimization, which is needed to train deep neural networks. The unified algorithm includes the existing adaptive-learning-rate optimization algorithms, such as Adaptive Moment Estimation (Adam), Adaptive Mean Square Gradie...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE access Ročník 9; s. 143807 - 143823
Hlavní autoři:	Zhu, Yini, Iiduka, Hideaki
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Piscataway IEEE 2021 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Adam Adaptive algorithms Adaptive learning adaptive-learning-rate optimization algorithm AMSGrad AMSGWDC Approximation algorithms Artificial neural networks Convergence Convex functions Deep learning deep neural network GWDC Heuristic algorithms heuristic intelligent optimization methods learning rate Machine learning Neural networks nonconvex stochastic optimization Optimization Optimization algorithms stationary point problem Symmetric matrices
ISSN:	2169-3536, 2169-3536
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Buďte první, kdo okomentuje tento záznam!