Adversarial principal component analysis

This paper studies the following question: where should an adversary place an outlier of a given magnitude in order to maximize the error of the subspace estimated by PCA? We give the exact location of this worst possible outlier, and the exact expression of the maximum possible error. Equivalently, we determine the information-theoretic bounds on how much an outlier can tilt a subspace in its direction. This in turn provides universal (worst-case) error bounds for PCA under arbitrary noisy settings. Our results also have several implications on adaptive PCA, online PCA, and rank-one updates. We illustrate our results with a subspace tracking experiment.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[3]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[4]  Weihua Li,et al.  Recursive PCA for Adaptive Process Monitoring , 1999 .

[5]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[6]  R. Maronna.  Robust $M$-Estimators of Multivariate Location and Scatter , 1976 .

[7]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[8]  Jiazhong Nie,et al.  Online PCA with Optimal Regrets , 2013, ALT.

[9]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[10]  George Atia,et al.  A Subspace Learning Approach for High Dimensional Matrix Decomposition with Efficient Column/Row Sampling , 2016, ICML.

[11]  S. Eisenstat,et al.  A Stable and Efficient Algorithm for the Rank-One Modification of the Symmetric Eigenproblem , 1994, SIAM J. Matrix Anal. Appl..

[12]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[13]  Deniz Erdogmus,et al.  Perturbation-Based Eigenvector Updates for On-Line Principal Components Analysis and Canonical Correlation Analysis , 2006, J. VLSI Signal Process..

[14]  Christos Boutsidis,et al.  Online Principal Components Analysis , 2015, SODA.

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[17]  Zohar S. Karnin,et al.  Online {PCA} with Spectral Bounds , 2015 .

[18]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[19]  Ameet Talwalkar,et al.  Divide-and-Conquer Matrix Factorization , 2011, NIPS.

[20]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[21]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[22]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Robert D. Nowak,et al.  Random Consensus Robust PCA , 2017, AISTATS.