Nuisance Parameter Estimation Algorithms for Separable Nonlinear Models

Many inverse problems in machine learning, system identification, and image processing include nuisance parameters, which are important for the recovering of other parameters. Separable nonlinear optimization problems fall into this category. The special separable structure in these problems has ins...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on systems, man, and cybernetics. Systems Ročník 52; číslo 11; s. 7236 - 7247
Hlavní autoři: Chen, Long, Chen, Jia-Bing, Chen, Guang-Yong, Gan, Min, Chen, C. L. Philip
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.11.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Approximation algorithms
Autoregressive processes
Convergence
Ear
Expectation-maximization (EM) algorithm
hierarchical identification algorithm
Image processing
Inverse problems
Jacobian matrices
Machine learning
Mathematical models
Monte Carlo methods
Nuisance
Optimization
Parameter estimation
Parameter identification
Robustness (mathematics)
separable nonlinear optimization problem
System identification
variable projection (VP) algorithm
ISSN:2168-2216, 2168-2232
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Many inverse problems in machine learning, system identification, and image processing include nuisance parameters, which are important for the recovering of other parameters. Separable nonlinear optimization problems fall into this category. The special separable structure in these problems has inspired several efficient optimization strategies. A well-known method is the variable projection (VP) that projects out a subset of the estimated parameters, resulting in a reduced problem that includes fewer parameters. The expectation maximization (EM) is another separated method that provides a powerful framework for the estimation of nuisance parameters. The relationships between EM and VP were ignored in previous studies, though they deal with a part of parameters in a similar way. In this article, we explore the internal relationships and differences between VP and EM. Unlike the algorithms that separate the parameters directly, the hierarchical identification algorithm decomposes a complex model into several linked submodels and identifies the corresponding parameters. Therefore, this article also studies the difference and connection between the hierarchical algorithm and the parameter-separated algorithms like VP and EM. In the numerical simulation part, Monte Carlo experiments are performed to further compare the performance of different algorithms. The results show that the VP algorithm usually converges faster than the other two algorithms and is more robust to the initial point of the parameters.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2168-2216
2168-2232
DOI:10.1109/TSMC.2022.3155871