The influence of parameter initialization on the training time and accuracy of a nonlinear regression model

In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Journal of communications technology & electronics Ročník 61; číslo 6; s. 646 - 660
Hlavní autoři: Burnaev, E. V., Erofeev, P. D.
Médium: Journal Article
Jazyk:angličtina
Vydáno: Moscow Pleiades Publishing 01.06.2016
Springer
Springer Nature B.V
Témata:
Accuracy
Algorithms
Approximation
Artificial neural networks
Communications Engineering
Communications equipment
Communications technology
Comparative analysis
Decomposition
Dependence
Dictionaries
Engineering
Experiments
Learning
Mathematical analysis
Mathematical models
Mathematical Models and Computational Methods
Mean square errors
Networks
Neural networks
Nonlinear systems
Nonlinearity
Regression
Regression analysis
Studies
Support vector machines
parameter initialization
nonlinear regression
approximation
neural networks
RProp algorithm
SCAWI algorithm
error back propagation algorithm
ISSN:1064-2269, 1555-6557
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented as a decomposition in a linear dictionary of some parametric functions (sigmoids). We proposed a general deterministic approach for initialization, providing repeatability of results, reduction of a learning time and in some cases increase of a regression model accuracy; we developed two new algorithms (based on a piecewise-linear approximation and based on local properties of approximable dependency) in the framework of the proposed approach; we developed randomized initialization algorithm (spherical initialization) for effective approximation of high-dimensional dependencies; we improved the classical initialization method SCAWI (by locating centers of sigmoids in sample points), providing a regression model accuracy improvement on specific classes of dependencies (smooth functions and discontinuous functions with a number of local peculiarities in an input domain) when using RProp algorithm for learning; we performed comparison of classical and newly proposed initialization methods and highlighted the most efficient ones.
Bibliografie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:1064-2269
1555-6557
DOI:10.1134/S106422691606005X