Grassmann Manifold Optimization for Fast $L_1$-Norm Principal Component Analysis

In this letter, we propose a fast Grassmann manifold optimization method for <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-norm based principal component analysis (GM-<inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-PCA). Our approach is a two-step iterative cost-minimization and manifold retraction technique that efficiently finds all principal components simultaneously. We perform complexity analysis and show that GM-<inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-PCA achieves a significant reduction in processing time while obtaining comparable or better results to current state-of-the-art <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-PCA methods. We further demonstrate the improvement of GM-<inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-PCA technique over <inline-formula><tex-math notation="LaTeX">$L_2$</tex-math></inline-formula>-PCA on a dataset of facial imagery corrupted with outlying data points. Our experiments show that GM-<inline-formula><tex-math notation="LaTeX">$L_1$</tex-math></inline-formula>-PCA is computationally more efficient and produces results with lower reprojection error than previous methods. Furthermore, the processing time of our approach is relatively independent of dataset size and well suited for various big-data problems commonly encountered today.

[1]  Panos P. Markopoulos,et al.  Some Options for L1-subspace Signal Processing , 2013, ISWCS.

[2]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[3]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[4]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[6]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[7]  T. Kanade,et al.  Robust subspace computation using L1 norm , 2003 .

[8]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[9]  Junbin Gao,et al.  Image Outlier Detection and Feature Extraction via L1-Norm-Based 2D Probabilistic PCA , 2015, IEEE Transactions on Image Processing.

[10]  Panos P. Markopoulos,et al.  Efficient L1-Norm Principal-Component Analysis via Bit Flipping , 2016, IEEE Transactions on Signal Processing.

[11]  Ying Liu,et al.  Face recognition with L1-norm subspaces , 2016, SPIE Commercial + Scientific Sensing and Imaging.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  José H. Dulá,et al.  The L1-norm best-fit hyperplane problem , 2013, Appl. Math. Lett..

[14]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[15]  Zoubin Ghahramani,et al.  Unifying linear dimensionality reduction , 2014, 1406.0873.

[16]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[17]  J. Brooks,et al.  A Pure L1-norm Principal Component Analysis. , 2013, Computational statistics & data analysis.

[18]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[19]  Dimitris A. Pados,et al.  Compressed-Sensed-Domain L1-PCA Video Surveillance , 2016, IEEE Transactions on Multimedia.

[20]  Panos P. Markopoulos,et al.  Fast computation of the L1-principal component of real-valued data , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[22]  L. Bottou.  Stochastic Gradient Learning in Neural Networks , 1991 .

[23]  Andreas Savakis,et al.  Fast L1-eigenfaces for robust face recogntion , 2014, 2014 IEEE Western New York Image and Signal Processing Workshop (WNYISPW).

[24]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.