Streaming PCA with Many Missing Entries

We consider the streaming memory-constrained principal component analysis (PCA) problem with missing entries, where the available storage is linear in the dimensionality of the problem, and each vector has so many missing entries that matrix completion is not possible. SVD-based methods cannot work because of the memory constraint, while imputation-based updates fail when faced with too many erasures. For this problem, we propose a method based on a block power update approach introduced in [14]. We show on synthetic as well as benchmark data sets, that our approach outperforms existing approaches for streaming PCA by a significant margin for several interesting problem settings. We also consider the popular spiked covariance model with randomly missing entries, and obtain the first known global convergence guarantees for this problem. We show that our method converges to the true “spike” using a number of samples that is linear in the dimension of the data. Moreover, our memory requirement is also linear in the ambient dimension. Thus, both memory and sample complexity have optimal scaling with dimension.