MEDRoP: Memory-Efficient Dynamic Robust PCA

Robust PCA (RPCA) is the problem of separating a given data matrix into the sum of a sparse matrix and a low-rank matrix. The column span of the low-rank matrix gives the PCA solution. Dynamic RPCA is the time-varying extension of RPCA. It assumes that the true data vectors lie in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. We propose an algorithm that we call Memory-Efficient Dynamic Robust PCA (MEDRoP). This relies on the recently studied recursive projected compressive sensing (ReProCS) framework for solving dynamic RPCA problems, however, the actual algorithm is significantly different from, and simpler than, previous ReProCS-based methods. The main contribution of this work is a theoretical guarantee that MEDRoP provably solves dynamic RPCA under weakened versions of standard RPCA assumptions, a mild assumption on slow subspace change, and two simple assumptions (a lower bound on most outlier magnitudes and mutual independence of the true data vectors). Our result is important because (i) it removes the strong assumptions needed by the three previous complete guarantees for ReProCS-based algorithms; (ii) it shows that, it is possible to achieve significantly improved outlier tolerance compared to static RPCA solutions by exploiting slow subspace change and a lower bound on most outlier magnitudes; (iii) it is able to track a changed subspace within a delay that is more than the subspace dimension by only logarithmic factors and thus is near-optimal; and (iv) it studies an algorithm that is online (after initialization), fast, and, memory-efficient (its memory complexity is within logarithmic factors of the optimal).