Tradeoffs Between Convergence Rate and Noise Amplification for Momentum-Based Accelerated Optimization Algorithms

In this article, we study momentum-based first-order optimization algorithms in which the iterations utilize information from the two previous steps and are subject to an additive white noise. This setup uses noise to account for uncertainty in either gradient evaluation or iteration updates, and it...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on automatic control Ročník 70; číslo 2; s. 889 - 904
Hlavní autori:	Mohammadi, Hesameddin, Razaviyayn, Meisam, Jovanovic, Mihailo R.
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.02.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Algorithms Convergence Convergence rate convex optimization Convexity first-order algorithms heavy-ball method Heuristic algorithms Lower bounds Momentum Nesterov's accelerated algorithm Noise noise amplification Optimization Optimization algorithms Parameter identification Parameter uncertainty Pareto optimum performance tradeoffs Settling settling time Stability criteria Steady-state Stochastic processes Tradeoffs Uncertainty Uncertainty principles Variance White noise
ISSN:	0018-9286, 1558-2523
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Buďte prvý, kto okomentuje tento záznam!