An Introduction to Development of Centralized and Distributed Stochastic Approximation Algorithm with Expanding Truncations

The stochastic approximation algorithm (SAA), starting from the pioneer work by Robbins and Monro in 1950s, has been successfully applied in systems and control, statistics, machine learning, and so forth. In this paper, we will review the development of SAA in China, to be specific, the stochastic...

Full description

Saved in:
Bibliographic Details
Published in:Algorithms Vol. 14; no. 6; p. 174
Main Author: Zhao, Wenxiao
Format: Journal Article
Language:English
Published: Basel MDPI AG 01.06.2021
Subjects:
Algorithms
Approximation
centralized and distributed algorithm
Differential equations
expanding truncation technique
Machine learning
Mathematical analysis
Noise
Principal components analysis
Probabilistic methods
stochastic approximation algorithm
ISSN:1999-4893, 1999-4893
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The stochastic approximation algorithm (SAA), starting from the pioneer work by Robbins and Monro in 1950s, has been successfully applied in systems and control, statistics, machine learning, and so forth. In this paper, we will review the development of SAA in China, to be specific, the stochastic approximation algorithm with expanding truncations (SAAWET) developed by Han-Fu Chen and his colleagues during the past 35 years. We first review the historical development for the centralized algorithm including the probabilistic method (PM) and the ordinary differential equation (ODE) method for SAA and the trajectory-subsequence method for SAAWET. Then, we will give an application example of SAAWET to the recursive principal component analysis. We will also introduce the recent progress on SAAWET in a networked and distributed setting, named the distributed SAAWET (DSAAWET).
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1999-4893
1999-4893
DOI:10.3390/a14060174