Insights Into Algorithms for Separable Nonlinear Least Squares Problems

Separable nonlinear least squares (SNLLS) problems have attracted interest in a wide range of research fields such as machine learning, computer vision, and signal processing. During the past few decades, several algorithms, including the joint optimization algorithm, alternated least squares (ALS)...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing Vol. 30; pp. 1207 - 1218
Main Authors: Chen, Guang-Yong, Gan, Min, Wang, Shuqiang, Chen, C. L. Philip
Format: Journal Article
Language:English
Published: United States IEEE 2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Analytical models
Autoregressive models
Autoregressive processes
Bundle adjustment
Computer vision
Image restoration
Jacobian matrices
Least squares
Machine learning
Optimization
parameter estimation
Partitioning algorithms
Radial basis function
RBF-AR model
Robustness
Robustness (mathematics)
separable nonlinear least-squares problem
Signal processing
Signal processing algorithms
Variable projection
ISSN:1057-7149, 1941-0042, 1941-0042
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Separable nonlinear least squares (SNLLS) problems have attracted interest in a wide range of research fields such as machine learning, computer vision, and signal processing. During the past few decades, several algorithms, including the joint optimization algorithm, alternated least squares (ALS) algorithm, embedded point iterations (EPI) algorithm, and variable projection (VP) algorithms, have been employed for solving SNLLS problems in the literature. The VP approach has been proven to be quite valuable for SNLLS problems and the EPI method has been successful in solving many computer vision tasks. However, no clear explanations about the intrinsic relationships of these algorithms have been provided in the literature. In this paper, we give some insights into these algorithms for SNLLS problems. We derive the relationships among different forms of the VP algorithms, EPI algorithm and ALS algorithm. In addition, the convergence and robustness of some algorithms are investigated. Moreover, the analysis of the VP algorithm generates a negative answer to Kaufman's conjecture. Numerical experiments on the image restoration task, fitting the time series data using the radial basis function network based autoregressive (RBF-AR) model, and bundle adjustment are given to compare the performance of different algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1057-7149
1941-0042
1941-0042
DOI:10.1109/TIP.2020.3043087