A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems

Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNL...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control Vol. 64; no. 2; pp. 526 - 537
Main Authors: Chen, Guang-Yong, Gan, Min, Chen, C. L. Philip, Li, Han-Xiong
Format: Journal Article
Language:English
Published: New York IEEE 01.02.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Approximation algorithms
Conditioning
Data fitting
Gallium nitride
Iterative methods
Jacobian matrices
Least squares
Linear programming
Machine learning
Machine learning algorithms
Optimization
Parameter estimation
Regularization
separable nonlinear least squares (SNLLS)
Statistical models
System identification
variable projection (VP)
weighted generalized cross validation (WGCV)
ISSN:0018-9286, 1558-2523
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNLLS problems based on the VP method. Selecting an appropriate regularization parameter is difficult because of the nonlinear optimization procedure. We propose to determine the regularization parameter using the weighted generalized cross-validation method at every iteration. This makes the original objective function changing during the optimization procedure. To circumvent this problem, we use an inequation to produce a consistent demand of decreasing at successive iterations. The approximation of the Jacobian of the regularized problem is also discussed. The proposed regularized VP algorithm is tested by the parameter estimation problem of several statistical models. Numerical results demonstrate the effectiveness of the proposed algorithm.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2018.2838045