Compressive PCA for Low-Rank Matrices on Graphs

We introduce a novel framework for an approximate recovery of data matrices which are low rank on graphs, from sampled measurements. The rows and columns of such matrices belong to the span of the first few eigenvectors of the graphs constructed between their rows and columns. We leverage this property to recover the nonlinear low-rank structures efficiently from sampled data measurements, with a low cost (linear in <inline-formula> <tex-math notation="LaTeX">$n$</tex-math></inline-formula>). First, a resrtricted isometry property condition is introduced for efficient uniform sampling of the rows and columns of such matrices based on the cumulative coherence of graph eigenvectors. Second, a state-of-the-art fast low-rank recovery method is suggested for the sampled data. Finally, several efficient, parallel, and parameter-free decoders are presented along with their theoretical analysis for decoding the low-rank and cluster indicators for the full data matrix. Thus, we overcome the computational limitations of the standard <italic>linear</italic> low-rank recovery methods for big datasets. Our method can also be seen as a major step toward efficient recovery of nonlinear low-rank structures. For a matrix of size <inline-formula> <tex-math notation="LaTeX">$n \times p$</tex-math></inline-formula>, on a single core machine, our method gains a speed up of <inline-formula><tex-math notation="LaTeX">$p^2/k$</tex-math></inline-formula> over robust principal component analysis (RPCA), where <inline-formula><tex-math notation="LaTeX">$k \ll p$</tex-math></inline-formula> is the subspace dimension. Numerically, we can recover a low-rank matrix of size <inline-formula><tex-math notation="LaTeX"> $10304 \times 1000$</tex-math></inline-formula>, 100 times faster than RPCA.