The influence of parameter initialization on the training time and accuracy of a nonlinear regression model

In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented...

Full description

Saved in:
Bibliographic Details
Published in:Journal of communications technology & electronics Vol. 61; no. 6; pp. 646 - 660
Main Authors: Burnaev, E. V., Erofeev, P. D.
Format: Journal Article
Language:English
Published: Moscow Pleiades Publishing 01.06.2016
Springer
Springer Nature B.V
Subjects:
Accuracy
Algorithms
Approximation
Artificial neural networks
Communications Engineering
Communications equipment
Communications technology
Comparative analysis
Decomposition
Dependence
Dictionaries
Engineering
Experiments
Learning
Mathematical analysis
Mathematical models
Mathematical Models and Computational Methods
Mean square errors
Networks
Neural networks
Nonlinear systems
Nonlinearity
Regression
Regression analysis
Studies
Support vector machines
parameter initialization
nonlinear regression
approximation
neural networks
RProp algorithm
SCAWI algorithm
error back propagation algorithm
ISSN:1064-2269, 1555-6557
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented as a decomposition in a linear dictionary of some parametric functions (sigmoids). We proposed a general deterministic approach for initialization, providing repeatability of results, reduction of a learning time and in some cases increase of a regression model accuracy; we developed two new algorithms (based on a piecewise-linear approximation and based on local properties of approximable dependency) in the framework of the proposed approach; we developed randomized initialization algorithm (spherical initialization) for effective approximation of high-dimensional dependencies; we improved the classical initialization method SCAWI (by locating centers of sigmoids in sample points), providing a regression model accuracy improvement on specific classes of dependencies (smooth functions and discontinuous functions with a number of local peculiarities in an input domain) when using RProp algorithm for learning; we performed comparison of classical and newly proposed initialization methods and highlighted the most efficient ones.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:1064-2269
1555-6557
DOI:10.1134/S106422691606005X