Subexponential-Time Algorithms for Sparse PCA

We study the computational cost of recovering a unit-norm sparse principal component x ∈ R n planted in a random matrix, in either the Wigner or Wishart spiked model (observing either W + λ x x ⊤ with W drawn from the Gaussian orthogonal ensemble, or N independent samples from N ( 0 , I n + β x x ⊤...

Full description

Saved in:
Bibliographic Details
Published in:Foundations of computational mathematics Vol. 24; no. 3; pp. 865 - 914
Main Authors: Ding, Yunzi, Kunisky, Dmitriy, Wein, Alexander S., Bandeira, Afonso S.
Format: Journal Article
Language:English
Published: New York Springer US 01.06.2024
Springer Nature B.V
Subjects:
Algorithms
Applications of Mathematics
Computational efficiency
Computer Science
Economics
Likelihood ratio
Linear and Multilinear Algebras
Math Applications in Computer Science
Mathematics
Mathematics and Statistics
Matrix Theory
Numerical Analysis
Polynomials
Principal components analysis
Recovery
Search algorithms
Signal to noise ratio
Tradeoffs
Low-degree likelihood ratio
Sparse PCA
Computational complexity
62F03
68Q87
68Q17
ISSN:1615-3375, 1615-3383
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the computational cost of recovering a unit-norm sparse principal component x ∈ R n planted in a random matrix, in either the Wigner or Wishart spiked model (observing either W + λ x x ⊤ with W drawn from the Gaussian orthogonal ensemble, or N independent samples from N ( 0 , I n + β x x ⊤ ) , respectively). Prior work has shown that when the signal-to-noise ratio ( λ or β N / n , respectively) is a small constant and the fraction of nonzero entries in the planted vector is ‖ x ‖ 0 / n = ρ , it is possible to recover x in polynomial time if ρ ≲ 1 / n . While it is possible to recover x in exponential time under the weaker condition ρ ≪ 1 , it is believed that polynomial-time recovery is impossible unless ρ ≲ 1 / n . We investigate the precise amount of time required for recovery in the “possible but hard” regime 1 / n ≪ ρ ≪ 1 by exploring the power of subexponential-time algorithms, i.e., algorithms running in time exp ( n δ ) for some constant δ ∈ ( 0 , 1 ) . For any 1 / n ≪ ρ ≪ 1 , we give a recovery algorithm with runtime roughly exp ( ρ 2 n ) , demonstrating a smooth tradeoff between sparsity and runtime. Our family of algorithms interpolates smoothly between two existing algorithms: the polynomial-time diagonal thresholding algorithm and the exp ( ρ n ) -time exhaustive search algorithm. Furthermore, by analyzing the low-degree likelihood ratio, we give rigorous evidence suggesting that the tradeoff achieved by our algorithms is optimal.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1615-3375
1615-3383
DOI:10.1007/s10208-023-09603-0