Tradeoffs Between Convergence Rate and Noise Amplification for Momentum-Based Accelerated Optimization Algorithms

In this article, we study momentum-based first-order optimization algorithms in which the iterations utilize information from the two previous steps and are subject to an additive white noise. This setup uses noise to account for uncertainty in either gradient evaluation or iteration updates, and it...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on automatic control Vol. 70; no. 2; pp. 889 - 904
Main Authors:	Mohammadi, Hesameddin, Razaviyayn, Meisam, Jovanovic, Mihailo R.
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.02.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Convergence Convergence rate convex optimization Convexity first-order algorithms heavy-ball method Heuristic algorithms Lower bounds Momentum Nesterov's accelerated algorithm Noise noise amplification Optimization Optimization algorithms Parameter identification Parameter uncertainty Pareto optimum performance tradeoffs Settling settling time Stability criteria Steady-state Stochastic processes Tradeoffs Uncertainty Uncertainty principles Variance White noise
ISSN:	0018-9286, 1558-2523
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!