Tradeoffs Between Convergence Rate and Noise Amplification for Momentum-Based Accelerated Optimization Algorithms

In this article, we study momentum-based first-order optimization algorithms in which the iterations utilize information from the two previous steps and are subject to an additive white noise. This setup uses noise to account for uncertainty in either gradient evaluation or iteration updates, and it...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on automatic control Jg. 70; H. 2; S. 889 - 904
Hauptverfasser:	Mohammadi, Hesameddin, Razaviyayn, Meisam, Jovanovic, Mihailo R.
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.02.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Algorithms Convergence Convergence rate convex optimization Convexity first-order algorithms heavy-ball method Heuristic algorithms Lower bounds Momentum Nesterov's accelerated algorithm Noise noise amplification Optimization Optimization algorithms Parameter identification Parameter uncertainty Pareto optimum performance tradeoffs Settling settling time Stability criteria Steady-state Stochastic processes Tradeoffs Uncertainty Uncertainty principles Variance White noise
ISSN:	0018-9286, 1558-2523
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Schreiben Sie den ersten Kommentar!