Low Rank Approximation and Decomposition of Large Matrices Using Error Correcting Codes

Low rank approximation is an important tool used in many applications of signal processing and machine learning. Recently, randomized sketching algorithms were proposed to effectively construct low rank approximations and obtain approximate singular value decompositions of large matrices. Similar id...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on information theory Ročník 63; číslo 9; s. 5544 - 5558
Hlavní autoři: Ubaru, Shashanka, Mazumdar, Arya, Saad, Yousef
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.09.2017
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithm design and analysis
Approximation
Approximation algorithms
Columns (structural)
Decomposition
Error correcting codes
Error correction
Error correction & detection
Error correction codes
Least squares method
low rank approximation
Machine learning
Mathematical analysis
Matrix decomposition
Matrix methods
randomized sketching algorithms
Regression
Regression analysis
Signal processing
Sparse matrices
Statistical analysis
subspace embedding
Transforms
ISSN:0018-9448, 1557-9654
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Low rank approximation is an important tool used in many applications of signal processing and machine learning. Recently, randomized sketching algorithms were proposed to effectively construct low rank approximations and obtain approximate singular value decompositions of large matrices. Similar ideas were used to solve least squares regression problems. In this paper, we show how matrices from error correcting codes can be used to find such low rank approximations and matrix decompositions, and extend the framework to linear least squares regression problems. The benefits of using these code matrices are the following. First, they are easy to generate and they reduce randomness significantly. Second, code matrices, with mild restrictions, satisfy the subspace embedding property, and have a better chance of preserving the geometry of an large subspace of vectors. Third, for parallel and distributed applications, code matrices have significant advantages over structured random matrices and Gaussian random matrices. Fourth, unlike Fourier or Hadamard transform matrices, which require sampling O(k log k) columns for a rank-k approximation, the log factor is not necessary for certain types of code matrices. In particular, (1 + ϵ) optimal Frobenius norm error can be achieved for a rank-k approximation with O(k/ϵ) samples. Fifth, fast multiplication is possible with structured code matrices, so fast approximations can be achieved for general dense input matrices. Sixth, for least squares regression problem min ∥Ax - b∥ 2 where A ∈ R n×d , the (1 + E) relative error approximation can be achieved with O(d/ϵ) samples, with high probability, when certain code matrices are used.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2017.2723898