Sparse Approximation to the Eigensubspace for Discrimination

Two-dimensional (2-D) image-matrix-based projection methods for feature extraction are widely used in many fields of computer vision and pattern recognition. In this paper, we propose a novel framework called sparse 2-D projections (S2DP) for image feature extraction. Different from the existing 2-D feature extraction methods, S2DP iteratively learns the sparse projection matrix by using elastic net regression and singular value decomposition. Theoretical analysis shows that the optimal sparse subspace approximates the eigensubspace obtained by solving the corresponding generalized eigenequation. With the S2DP framework, many 2-D projection methods can be easily extended to sparse cases. Moreover, when each row/column of the image matrix is regarded as an independent high-dimensional vector (1-D vector), it is proven that the vector-based eigensubspace is also approximated by the sparse subspace obtained by the same method used in this paper. Theoretical analysis shows that, when compared with the vector-based sparse projection learning methods, S2DP greatly saves both computation and memory costs. This property makes S2DP more tractable for real-world applications. Experiments on well-known face databases indicate the competitive performance of the proposed S2DP over some 2-D projection methods when facial expressions, lighting conditions, and time vary.

[1]  I. Jolliffe.  Principal Component Analysis , 2002 .

[2]  Stefanos Zafeiriou,et al.  Regularized Kernel Discriminant Analysis With a Robust Kernel for Face Recognition and Verification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Jian Yang,et al.  An approach for directly extracting features from matrix data and its application in face recognition , 2008, Neurocomputing.

[4]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[5]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[6]  Shai Avidan,et al.  Generalized spectral bounds for sparse LDA , 2006, ICML.

[7]  Jianhua Z. Huang,et al.  Sparse Linear Discriminant Analysis with Applications to High Dimensional Low Sample Size Data , 2009 .

[8]  Simon C. K. Shiu,et al.  Two Dimensional Laplacianfaces Method for Face Recognition , 2006, RSCTC.

[9]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[10]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2004, Machine Learning.

[11]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[13]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[14]  Zhonghua Liu,et al.  Two-Dimensional Local Graph Embedding Discriminant Analysis(F2DLGEDA) with Its Application to Face and Palm Biometrics , 2009 .

[15]  Deng Cai,et al.  Tensor Subspace Analysis , 2005, NIPS.

[16]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[17]  Yong Xu,et al.  One improvement to two-dimensional locality preserving projection method for use with face recognition , 2009, Neurocomputing.

[18]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[19]  Fengxi Song,et al.  A novel local preserving projection scheme for use with face recognition , 2010, Expert Syst. Appl..

[20]  Xindong Wu,et al.  Manifold elastic net: a unified framework for sparse dimension reduction , 2010, Data Mining and Knowledge Discovery.

[21]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[22]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[23]  Jieping Ye,et al.  Two-Dimensional Linear Discriminant Analysis , 2004, NIPS.

[24]  Kuanquan Wang,et al.  Bidirectional PCA with assembled matrix distance metric for image recognition , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Bin Luo,et al.  2D-LPP: A two-dimensional extension of locality preserving projections , 2007, Neurocomputing.

[26]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  L. Eldén.  Algorithms for the regularization of ill-conditioned least squares problems , 1977 .

[28]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[30]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[31]  Zhong Jin,et al.  Two-Dimensional Local Graph Embedding Discriminant Analysis(F2DLGEDA) with Its Application to Face and Palm Biometrics , 2009, 2009 Chinese Conference on Pattern Recognition.

[32]  Dong Xu,et al.  Discriminant analysis with tensor representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Hua Huang,et al.  Super-Resolution Method for Face Recognition Using Nonlinear Mappings on Coherent Features , 2011, IEEE Transactions on Neural Networks.

[34]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[35]  Liwei Wang,et al.  On image matrix based feature extraction algorithms , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[37]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[39]  Dewen Hu,et al.  Comment on: "Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition" , 2008, Pattern Recognit..

[40]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[41]  BakerSimon,et al.  The CMU Pose, Illumination, and Expression Database , 2003 .

[42]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[43]  David Zhang,et al.  Local Linear Discriminant Analysis Framework Using Sample Neighbors , 2011, IEEE Transactions on Neural Networks.

[44]  Zhonglong Zheng.  Sparse Locality Preserving Embedding , 2009, 2009 2nd International Congress on Image and Signal Processing.

[45]  Qiuqi Ruan,et al.  Facial expression recognition based on two-dimensional discriminant locality preserving projections , 2008, Neurocomputing.

[46]  Shai Avidan,et al.  Spectral Bounds for Sparse PCA: Exact and Greedy Algorithms , 2005, NIPS.

[47]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[48]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[49]  H. Zou,et al.  Regression Shrinkage and Selection via the Elastic Net , with Applications to Microarrays , 2003 .

[50]  Jian Yang,et al.  Sparse Tensor Discriminant Color Space for Face Verification , 2012, IEEE Transactions on Neural Networks and Learning Systems.