Iteratively Reweighted Least Squares Algorithms for L1-Norm Principal Component Analysis

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of the reconstructed data, we propose an exact reweighted and an approximate algorithm based on iteratively reweighted least squares. We provide convergence analyses, and compare their performance against benchmark algorithms in the literature. The computational experiment shows that the proposed algorithms consistently perform best.

[1]  Guoying Li,et al.  Projection-Pursuit Approach to Robust Dispersion Matrices and Principal Components: Primary Theory and Monte Carlo , 1985 .

[2]  J. Brooks,et al.  A Pure L1-norm Principal Component Analysis. , 2013, Computational statistics & data analysis.

[3]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[6]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[7]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[9]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[10]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[11]  D. Hawkins,et al.  Methods of L1 estimation of a covariance matrix , 1987 .

[12]  Christophe Croux,et al.  High breakdown estimators for principal components: the projection-pursuit approach revisited , 2005 .

[13]  Amir Averbuch,et al.  Updating kernel methods in spectral decomposition by affinity perturbations , 2012 .

[14]  Vartan Choulakian,et al.  L1-norm projection pursuit principal component analysis , 2006, Comput. Stat. Data Anal..

[15]  J. Paul Brooks,et al.  pcaL 1 : An Implementation in R of Three Methods for L 1-Norm Principal Component Analysis , 2013 .

[16]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[17]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[18]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[19]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[20]  Murray A. Jorgensen.  Iteratively Reweighted Least Squares , 2006 .

[21]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[22]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Philippe C. Besse,et al.  A L 1-norm PCA and a Heuristic Approach , 1996 .

[24]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.