A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems

Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNL...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on automatic control Ročník 64; číslo 2; s. 526 - 537
Hlavní autoři: Chen, Guang-Yong, Gan, Min, Chen, C. L. Philip, Li, Han-Xiong
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.02.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Approximation algorithms
Conditioning
Data fitting
Gallium nitride
Iterative methods
Jacobian matrices
Least squares
Linear programming
Machine learning
Machine learning algorithms
Optimization
Parameter estimation
Regularization
separable nonlinear least squares (SNLLS)
Statistical models
System identification
variable projection (VP)
weighted generalized cross validation (WGCV)
ISSN:0018-9286, 1558-2523
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNLLS problems based on the VP method. Selecting an appropriate regularization parameter is difficult because of the nonlinear optimization procedure. We propose to determine the regularization parameter using the weighted generalized cross-validation method at every iteration. This makes the original objective function changing during the optimization procedure. To circumvent this problem, we use an inequation to produce a consistent demand of decreasing at successive iterations. The approximation of the Jacobian of the regularized problem is also discussed. The proposed regularized VP algorithm is tested by the parameter estimation problem of several statistical models. Numerical results demonstrate the effectiveness of the proposed algorithm.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2018.2838045