Generalized two-dimensional PCA based on ℓ 2, p-norm minimization

To exploit the information from two-dimensional structured data, two-dimensional principal component analysis (2-DPCA) has been widely used for dimensionality reduction and feature extraction. However, 2-DPCA is sensitive to outliers which are common in real applications. Therefore, many robust 2-DPCA methods have been proposed to improve the robustness of 2-DPCA. But existing robust 2-DPCAs have several weaknesses. First, these methods cannot be robust enough to outliers. Second, to center a sample set mixed with outliers using the L2-norm distance is usually biased. Third, most methods do not preserve the nice property of 2-DPCA (rotational invariance), which is important for learning algorithm. To alleviate these issues, we present a generalized robust 2-DPCA, which is named as 2-DPCA with $$\ell _{2,p}$$ -norm minimization ( $$\ell _{2,p}$$ -2-DPCA), for image representation and recognition. In $$\ell _{2,p}$$ -2-DPCA, $$\ell _{2,p}$$ -norm is employed as the distance metric to measure the reconstruction error, which can alleviate the effect of outliers. Therefore, the proposed method is robust to outliers and preserves the desirable property of 2-DPCA which is invariant to rotational and well characterizes the geometric structure of samples. Moreover, most existing robust PCA methods estimate sample mean from database with outliers by averaging, which is usually biased. Sample mean are treated as an unknown variable to remedy the bias of computing sample mean in $$\ell _{2,p}$$ -2-DPCA. To solve $$\ell _{2,p}$$ -2-DPCA, we propose an iterative algorithm, which has a closed-form solution in each iteration. Experimental results on several benchmark databases demonstrate the effectiveness and advantages of our method.

[1]  Lei Wang,et al.  Generalized 2D principal component analysis for face image representation and recognition , 2005, Neural Networks.

[2]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[3]  Yudong Chen,et al.  Robust Discriminative Principal Component Analysis , 2018, CCBR.

[4]  Wen Gao,et al.  Just Noticeable Difference Estimation for Screen Content Images , 2016, IEEE Transactions on Image Processing.

[5]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[6]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Can Gao,et al.  Nuclear norm based two-dimensional sparse principal component analysis , 2018, Int. J. Wavelets Multiresolution Inf. Process..

[8]  Zhihui Lai,et al.  Principal Component Analysis based on Nuclear norm Minimization , 2019, Neural Networks.

[9]  Feiping Nie,et al.  $\ell _{2,p}$ -Norm Based PCA for Image Recognition , 2018, IEEE Transactions on Image Processing.

[10]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Zhenhua Guo,et al.  A Framework of Joint Graph Embedding and Sparse Regression for Dimensionality Reduction , 2015, IEEE Transactions on Image Processing.

[12]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[13]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  Zhenyu He,et al.  Joint sparse principal component analysis , 2017, Pattern Recognit..

[15]  Yong Xu,et al.  RPCA-Based Tumor Classification Using Gene Expression Data , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[16]  Jian Yang,et al.  Nuclear Norm-Based 2-DPCA for Extracting Features From Images , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Diego Klabjan,et al.  Iteratively Reweighted Least Squares Algorithms for L1-Norm Principal Component Analysis , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[18]  Xuelong Li,et al.  Nuclear Norm-Based 2DLPP for Image Classification , 2017, IEEE Transactions on Multimedia.

[19]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[20]  Panos P. Markopoulos,et al.  L1-Norm Principal-Component Analysis of Complex Data , 2017, IEEE Transactions on Signal Processing.

[21]  Ming Shao,et al.  Discriminative metric: Schatten norm vs. vector norm , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[22]  José H. Dulá,et al.  A pure L1L1-norm principal component analysis , 2013, Comput. Stat. Data Anal..

[23]  Zhenhua Guo,et al.  Face recognition by sparse discriminant analysis via joint L2, 1-norm minimization , 2014, Pattern Recognit..

[24]  Panos P. Markopoulos,et al.  Efficient L1-Norm Principal-Component Analysis via Bit Flipping , 2016, IEEE Transactions on Signal Processing.

[25]  Qianqian Wang,et al.  Two-Dimensional PCA with F-Norm Minimization , 2017, AAAI.

[26]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[27]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[28]  Zhenhua Guo,et al.  Two-Dimensional Whitening Reconstruction for Enhancing Robustness of Principal Component Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Chengqi Zhang,et al.  Convex Sparse PCA for Unsupervised Feature Learning , 2014, ACM Trans. Knowl. Discov. Data.

[30]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[32]  Zhihui Lai,et al.  Joint Sparse Neighborhood Preserving Embedding , 2019 .

[33]  Zhenhua Guo,et al.  Active learning via local structure reconstruction , 2017, Pattern Recognit. Lett..

[34]  Zhenhua Guo,et al.  Self-learning for face clustering , 2018, Pattern Recognit..

[35]  Philippe C. Besse,et al.  A L 1-norm PCA and a Heuristic Approach , 1996 .

[36]  Jing Wang,et al.  2DPCA with L1-norm for simultaneously robust and sparse modelling , 2013, Neural Networks.

[37]  Ye Zhang,et al.  Sparse Nuclear Norm Two Dimensional Principal Component Analysis , 2016, CCBR.

[38]  Nojun Kwak,et al.  Principal Component Analysis by $L_{p}$ -Norm Maximization , 2014, IEEE Transactions on Cybernetics.

[39]  Zhenhua Guo,et al.  Structured orthogonal matching pursuit for feature selection , 2019, Neurocomputing.

[40]  Zhenhua Guo,et al.  Robust principal component analysis via optimal mean by joint ℓ2, 1 and Schatten p-norms minimization , 2018, Neurocomputing.

[41]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[42]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[44]  A. Martínez,et al.  The AR face databasae , 1998 .

[45]  Fang Chen,et al.  ${R}_1$ -2-DPCA and Face Recognition , 2019, IEEE Transactions on Cybernetics.

[46]  Yong Wang,et al.  L1-norm-based principal component analysis with adaptive regularization , 2016, Pattern Recognit..

[47]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[48]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[50]  Feiping Nie,et al.  Compound Rank- $k$ Projections for Bilinear Analysis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Jian Yang,et al.  Rotational Invariant Dimensionality Reduction Algorithms , 2017, IEEE Transactions on Cybernetics.

[52]  Wai Keung Wong,et al.  Optimal Feature Selection for Robust Classification via l2,1-Norms Regularization , 2014, 2014 22nd International Conference on Pattern Recognition.

[53]  Jian Yang,et al.  Multilinear Sparse Principal Component Analysis , 2014, IEEE Transactions on Neural Networks and Learning Systems.