Adaptive L1-Norm Principal-Component Analysis With Online Outlier Rejection

L1-norm principal-component analysis (L1-PCA) is known to attain sturdy resistance against faulty points (outliers) among the processed data. However, computing the L1-PCA of large datasets, with high number of measurements and/or dimensions, may be computationally impractical; in such cases, incremental solutions could be preferred. At the same time, in many applications it is desired to track the signal subspace via principal component adaptation. In this paper, we present new methods for both incremental and adaptive L1-PCA. Our first algorithm computes L1-PCA incrementally, processing one measurement at a time and performing online rejection of possible outliers; due to its low computational and storage cost, this algorithm is appropriate for application to both large and streaming datasets. Our second algorithm combines the merits of the first one with the additional ability to track changes in the nominal signal subspace. The proposed algorithms are evaluated with experimental studies on subspace estimation/tracking, video surveillance, image conditioning, and direction-of-arrival estimation/tracking.

[1]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[2]  Brendt Wohlberg,et al.  Incremental Principal Component Pursuit for Video Background Modeling , 2015, Journal of Mathematical Imaging and Vision.

[3]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Sajid Javed,et al.  Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery , 2017, IEEE Signal Processing Magazine.

[5]  Namrata Vaswani,et al.  Static and Dynamic Robust PCA and Matrix Completion: A Review , 2018, Proceedings of the IEEE.

[6]  Feiping Nie,et al.  $\ell _{2,p}$ -Norm Based PCA for Image Recognition , 2018, IEEE Transactions on Image Processing.

[7]  B. S. Manjunath,et al.  An eigenspace update algorithm for image analysis , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[8]  Dimitris A. Pados,et al.  Video background tracking and foreground extraction via L1-subspace updates , 2016, SPIE Commercial + Scientific Sensing and Imaging.

[9]  Dimitris A. Pados,et al.  Optimal sparse L1-norm principal-component analysis , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Gonzalo Mateos,et al.  Robust PCA as Bilinear Decomposition With Outlier-Sparsity Regularization , 2011, IEEE Transactions on Signal Processing.

[11]  Stefanos Zafeiriou,et al.  Euler Principal Component Analysis , 2013, International Journal of Computer Vision.

[12]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[13]  Qinghua Zheng,et al.  Avoiding Optimal Mean ℓ2,1-Norm Maximization-Based Robust PCA for Reconstruction , 2017, Neural Computation.

[14]  Yueting Zhuang,et al.  Adaptive Weight Selection for Incremental Eigen-Background Modeling , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[15]  Panos P. Markopoulos,et al.  L1-Norm Principal-Component Analysis of Complex Data , 2017, IEEE Transactions on Signal Processing.

[16]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[17]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[18]  Dimitris A. Pados,et al.  Iterative re-weighted L1-norm principal-component analysis , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[19]  J. Bunch,et al.  Updating the singular value decomposition , 1978 .

[20]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[21]  Qinghua Zheng,et al.  Avoiding Optimal Mean Robust PCA/2DPCA with Non-greedy ℓ1-Norm Maximization , 2016, IJCAI.

[22]  Michael Langberg,et al.  L1-norm principal-component analysis in L2-norm-reduced-rank data subspaces , 2017, Commercial + Scientific Sensing and Imaging.

[23]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[24]  Erik G. Larsson,et al.  EVD-based channel estimation in multicell multiuser MIMO systems with very large antenna arrays , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[26]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[27]  Thierry Bouwmans,et al.  Subspace Learning for Background Modeling: A Survey , 2009 .

[28]  Namrata Vaswani,et al.  Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees , 2016, AISTATS.

[29]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[30]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[31]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[32]  Shie Mannor,et al.  Online PCA for Contaminated Data , 2013, NIPS.

[33]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[34]  Michael J. Black,et al.  Robust Principal Component Analysis for Computer Vision , 2001, ICCV.

[35]  Wen Gao,et al.  Robust Estimation of 3D Human Poses from a Single Image , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[37]  Panos P. Markopoulos,et al.  Indoor human motion classification by L1-norm subspaces of micro-Doppler signatures , 2017, 2017 IEEE Radar Conference (RadarConf).

[38]  Lei Wang,et al.  Adaptive Eigenbackground for Dynamic Background Modeling , 2006 .

[39]  O. Edfors,et al.  OFDM channel estimation by singular value decomposition , 1996, Proceedings of Vehicular Technology Conference - VTC.

[40]  Pablo Laguna,et al.  Principal Component Analysis in ECG Signal Processing , 2007, EURASIP J. Adv. Signal Process..

[41]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[43]  Tsagkarakis Nicholas,et al.  On the L1-Norm Approximation of a Matrix by Another of Lower Rank , 2016 .

[44]  Panos P. Markopoulos,et al.  Visual tracking with L1-Grassmann manifold modeling , 2017, Commercial + Scientific Sensing and Imaging.

[45]  Ying Liu,et al.  Compressed-Sensed-Domain L1-PCA Video Surveillance , 2015, IEEE Transactions on Multimedia.

[46]  J. Brooks,et al.  A Pure L1-norm Principal Component Analysis. , 2013, Computational statistics & data analysis.

[47]  Ka Yee Yeung,et al.  Principal component analysis for clustering gene expression data , 2001, Bioinform..

[48]  Vic Barnett,et al.  Outliers in Statistical Data , 1980 .

[49]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[50]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[51]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[52]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[53]  Jason Morphett,et al.  An integrated algorithm of incremental and robust PCA , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[54]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[55]  Chris H. Q. Ding,et al.  K-means clustering via principal component analysis , 2004, ICML.

[56]  Panos P. Markopoulos.  Reduced-rank filtering on L1-norm subspaces , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[57]  Yongmin Li,et al.  On incremental and robust subspace learning , 2004, Pattern Recognit..

[58]  Daniel K Sodickson,et al.  Low‐rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components , 2015, Magnetic resonance in medicine.

[59]  Morteza Mardani,et al.  Recovery of Low-Rank Plus Compressed Sparse Matrices With Application to Unveiling Traffic Anomalies , 2012, IEEE Transactions on Information Theory.

[60]  John C. S. Lui,et al.  Online Robust Subspace Tracking from Partial Information , 2011, ArXiv.

[61]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[62]  Zhi-Quan Luo,et al.  Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Namrata Vaswani,et al.  Provable Dynamic Robust PCA or Robust Subspace Tracking , 2017, 2018 IEEE International Symposium on Information Theory (ISIT).

[64]  Panos P. Markopoulos,et al.  Direction finding by complex L1-principal-component analysis , 2015, 2015 IEEE 16th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[65]  Bang Jun Lei,et al.  Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management , 2006, Pattern Recognit. Lett..

[66]  Ran He,et al.  Robust Principal Component Analysis Based on Maximum Correntropy Criterion , 2011, IEEE Transactions on Image Processing.

[67]  Panos P. Markopoulos,et al.  L1-fusion: Robust linear-time image recovery from few severely corrupted copies , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[68]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[69]  Ying Liu,et al.  Face recognition with L1-norm subspaces , 2016, SPIE Commercial + Scientific Sensing and Imaging.

[70]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[71]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[72]  Panos P. Markopoulos,et al.  Efficient L1-Norm Principal-Component Analysis via Bit Flipping , 2016, IEEE Transactions on Signal Processing.

[73]  Danijel Skocaj,et al.  Incremental and robust learning of subspace representations , 2008, Image Vis. Comput..

[74]  Dimitris A. Pados,et al.  Cloud-assisted individual l1-pca face recognition using wavelet-domain compressed images , 2016, 2016 6th European Workshop on Visual Information Processing (EUVIP).

[75]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[76]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[77]  Gilad Lerman,et al.  Robust Stochastic Principal Component Analysis , 2014, AISTATS.

[78]  Panos P. Markopoulos,et al.  Direction-of-arrival estimation by L1-norm principal components , 2016, 2016 IEEE International Symposium on Phased Array Systems and Technology (PAST).

[79]  Andreas Savakis,et al.  Fast L1-eigenfaces for robust face recogntion , 2014, 2014 IEEE Western New York Image and Signal Processing Workshop (WNYISPW).

[80]  Gilad Lerman,et al.  An Overview of Robust Subspace Recovery , 2018, Proceedings of the IEEE.

[81]  Morteza Mardani,et al.  Dynamic Anomalography: Tracking Network Anomalies Via Sparsity and Low Rank , 2012, IEEE Journal of Selected Topics in Signal Processing.