Adaptive stochastic approximation algorithm

In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Numerical algorithms Ročník 76; číslo 4; s. 917 - 937
Hlavní autori: Kresoja, Milena, Lužanin, Zorana, Stojkovska, Irena
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 01.12.2017
Springer Nature B.V
Predmet:
Adaptive algorithms
Algebra
Algorithms
Approximation
Computer Science
Iterative methods
Mathematical analysis
Mathematics
Numeric Computing
Numerical Analysis
Optimization techniques
Original Paper
Random variables
Theory of Computation
Unconstrained optimization
Adaptive step size
Stochastic approximation
Noisy function
Stochastic optimization
Gradient method
Descent direction
ISSN:1017-1398, 1572-9265
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1017-1398
1572-9265
DOI:10.1007/s11075-017-0290-4