The LASSO Risk for Gaussian Matrices

We consider the problem of learning a coefficient vector x ο ∈ R N from noisy linear observation y = Ax o + ∈ R n . In many contexts (ranging from model selection to image processing), it is desirable to construct a sparse estimator x̂. In this case, a popular approach consists in solving an ℓ 1 -pe...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on information theory Ročník 58; číslo 4; s. 1997 - 2017
Hlavní autoři:	Bayati, M., Montanari, A.
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York, NY IEEE 01.04.2012 Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Algorithm design and analysis Applied sciences Calibration Compressed sensing Detection, estimation, filtering, equalization, prediction Equations Exact sciences and technology graphical models Image processing Image processing systems Information theory Information, signal and communications theory Mean square error methods message passing algorithms Noise Noise measurement Normal distribution random matrix theory Services and terminals of telecommunications Signal and communications theory Signal processing Signal, noise Sparsity state evolution statistical learning Systems, networks and services of telecommunications Telecommunications Telecommunications and information theory Telemetry. Remote supervision. Telewarning. Remote control Vectors state evolution Range finding statistical learning Random matrix message passing algorithms Image processing Noise reduction Model selection Least squares problem Algorithm Mean square error Learning Message passing Algorithm performance Simulation graphical models Signal processing random matrix theory Compressed sensing
ISSN:	0018-9448, 1557-9654
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	We consider the problem of learning a coefficient vector x ο ∈ R N from noisy linear observation y = Ax o + ∈ R n . In many contexts (ranging from model selection to image processing), it is desirable to construct a sparse estimator x̂. In this case, a popular approach consists in solving an ℓ 1 -penalized least-squares problem known as the LASSO or basis pursuit denoising. For sequences of matrices A of increasing dimensions, with independent Gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic mean square error of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical model ideas. Simulations on real data matrices suggest that our results can be relevant in a broad array of practical applications.
Bibliografie:	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14
ISSN:	0018-9448 1557-9654
DOI:	10.1109/TIT.2011.2174612