Multi-stage stochastic gradient method with momentum acceleration

•Stage-wise optimization and momentum have been widely employed to accelerate SGD.•Negative momentum provides acceleration and stabilization on stochastic first-order methods.•Negative momentum extends Nesterovs momentum to the stage-wise optimization.•Gradient correction avoids the oscillations and...

Full description

Bibliographic Details

Published in:	Signal processing Vol. 188; p. 108201
Main Authors:	Luo, Zhijian, Chen, Siyu, Qian, Yuntao, Hou, Yueen
Format:	Journal Article
Language:	English
Published:	Elsevier B.V 01.11.2021
Subjects:	Convex optimization Momentum acceleration Multi-stage Stochastic gradient descent Stochastic gradient descent Convex optimization Multi-stage Momentum acceleration
ISSN:	0165-1684, 1872-7557
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!