Adaptive stochastic approximation algorithm

In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Numerical algorithms Jg. 76; H. 4; S. 917 - 937
Hauptverfasser: Kresoja, Milena, Lužanin, Zorana, Stojkovska, Irena
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.12.2017
Springer Nature B.V
Schlagworte:
Adaptive algorithms
Algebra
Algorithms
Approximation
Computer Science
Iterative methods
Mathematical analysis
Mathematics
Numeric Computing
Numerical Analysis
Optimization techniques
Original Paper
Random variables
Theory of Computation
Unconstrained optimization
Adaptive step size
Stochastic approximation
Noisy function
Stochastic optimization
Gradient method
Descent direction
ISSN:1017-1398, 1572-9265
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1017-1398
1572-9265
DOI:10.1007/s11075-017-0290-4