Phase Transitions and Sample Complexity in Bayes-Optimal Matrix Factorization

We analyze the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications, such as dictionary learning, blind matrix calibration, sparse principal component analysis, blind source separat...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on information theory Jg. 62; H. 7; S. 4228 - 4265
Hauptverfasser: Kabashima, Yoshiyuki, Krzakala, Florent, Mezard, Marc, Sakata, Ayaka, Zdeborova, Lenka
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.07.2016
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithm design and analysis
Algorithms
Bayesian analysis
Complexity theory
Computation
Factorization
Inference
Matrix
Noise measurement
Optimization
Phase transitions
Prediction algorithms
Principal component analysis
Probability distribution
Samples
Sparse matrices
Statistical analysis
Statistical mechanics
Statistical methods
statistical and computational tradeoff
statistical physics
message passing algorithms
phase transitions
computational barriers
Statistical inference
probabilistic matrix factorization
dictionary learning
sparse coding
ISSN:0018-9448, 1557-9654
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We analyze the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications, such as dictionary learning, blind matrix calibration, sparse principal component analysis, blind source separation, low rank matrix completion, robust principal component analysis, or factor analysis. It is also important in machine learning: unsupervised representation learning can often be studied through matrix factorization. We use the tools of statistical mechanics-the cavity and replica methods-to analyze the achievability and computational tractability of the inference problems in the setting of Bayes-optimal inference, which amounts to assuming that the two matrices have random-independent elements generated from some known distribution, and this information is available to the inference algorithm. In this setting, we compute the minimal mean-squared-error achievable, in principle, in any computational time, and the error that can be achieved by an efficient approximate message passing algorithm. The computation is based on the asymptotic state-evolution analysis of the algorithm. The performance that our analysis predicts, both in terms of the achieved mean-squared-error and in terms of sample complexity, is extremely promising and motivating for a further development of the algorithm.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2016.2556702