Tradeoffs Between Convergence Rate and Noise Amplification for Momentum-Based Accelerated Optimization Algorithms

In this article, we study momentum-based first-order optimization algorithms in which the iterations utilize information from the two previous steps and are subject to an additive white noise. This setup uses noise to account for uncertainty in either gradient evaluation or iteration updates, and it...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on automatic control Ročník 70; číslo 2; s. 889 - 904
Hlavní autoři:	Mohammadi, Hesameddin, Razaviyayn, Meisam, Jovanovic, Mihailo R.
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 01.02.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Algorithms Convergence Convergence rate convex optimization Convexity first-order algorithms heavy-ball method Heuristic algorithms Lower bounds Momentum Nesterov's accelerated algorithm Noise noise amplification Optimization Optimization algorithms Parameter identification Parameter uncertainty Pareto optimum performance tradeoffs Settling settling time Stability criteria Steady-state Stochastic processes Tradeoffs Uncertainty Uncertainty principles Variance White noise
ISSN:	0018-9286, 1558-2523
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Buďte první, kdo okomentuje tento záznam!