Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms

In this paper, we study the theoretical properties of iteratively re-weighted least squares (IRLS) algorithms and their utility in sparse signal recovery in the presence of noise. We demonstrate a one-to-one correspondence between the IRLS algorithms and a class of Expectation-Maximization (EM) algo...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on signal processing Ročník 62; číslo 1; s. 183 - 195
Hlavní autoři: Ba, Demba, Babadi, Behtash, Purdon, Patrick L., Brown, Emery N.
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, NY IEEE 01.01.2014
Institute of Electrical and Electronics Engineers
Témata:
Applied sciences
Compressive sampling
constrained maximum likelihood estimation
Convergence
Convex functions
Detection, estimation, filtering, equalization, prediction
Exact sciences and technology
expectation-maximization algorithms
Gaussian scale mixtures
Information, signal and communications theory
mathematical programming
Noise
Random variables
Sampling, quantization
Signal and communications theory
Signal processing algorithms
Signal, noise
Stability analysis
Telecommunications and information theory
Mobile radiocommunication
GSM system
expectation-maximization algorithms
Wireless telecommunication
Iterative method
Compressive sampling
Weighting
Gaussian process
Signal restoration
Gaussian scale mixtures
Simulation
Least squares method
constrained maximum likelihood estimation
Convergence rate
Signal processing
Signal reconstruction
Maximum likelihood
EM algorithm
Compressed sensing
Mathematical programming
ISSN:1053-587X, 1941-0476
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we study the theoretical properties of iteratively re-weighted least squares (IRLS) algorithms and their utility in sparse signal recovery in the presence of noise. We demonstrate a one-to-one correspondence between the IRLS algorithms and a class of Expectation-Maximization (EM) algorithms for constrained maximum likelihood estimation under a Gaussian scale mixture (GSM) distribution. The EM formalism, as well as the connection to GSMs, allow us to establish that the IRLS algorithms minimize smooth versions of the lν `norms', for . We leverage EM theory to show that the limit points of the sequence of IRLS iterates are stationary points of the smooth lν "norm" minimization problem on the constraint set. We employ techniques from Compressive Sampling (CS) theory to show that the IRLS algorithm is stable, if the limit point of the iterates coincides with the global minimizer. We further characterize the convergence rate of the IRLS algorithm, which implies global linear convergence for ν = 1 and local super-linear convergence for . We demonstrate our results via simulation experiments. The simplicity of IRLS, along with the theoretical guarantees provided in this contribution, make a compelling case for its adoption as a standard tool for sparse signal recovery.
ISSN:1053-587X
1941-0476
DOI:10.1109/TSP.2013.2287685