Adaptive stochastic approximation algorithm

In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure...

Full description

Saved in:
Bibliographic Details
Published in:Numerical algorithms Vol. 76; no. 4; pp. 917 - 937
Main Authors: Kresoja, Milena, Lužanin, Zorana, Stojkovska, Irena
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2017
Springer Nature B.V
Subjects:
Adaptive algorithms
Algebra
Algorithms
Approximation
Computer Science
Iterative methods
Mathematical analysis
Mathematics
Numeric Computing
Numerical Analysis
Optimization techniques
Original Paper
Random variables
Theory of Computation
Unconstrained optimization
Adaptive step size
Stochastic approximation
Noisy function
Stochastic optimization
Gradient method
Descent direction
ISSN:1017-1398, 1572-9265
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1017-1398
1572-9265
DOI:10.1007/s11075-017-0290-4