${L_{2,1}}$ -Norm Discriminant Manifold Learning

Recently, <inline-formula> <tex-math notation="LaTeX">${L_{1}}$ </tex-math></inline-formula>-norm-based robust discriminant feature extraction technique has been attracted much attention in dimensionality reduction and pattern recognition. However, it does not relate to the scatter matrix which well characterizes the geometric structure of data. In this paper, we propose a robust formulation of graph embedding framework for dimensionality reduction. In this robust framework, we use <inline-formula> <tex-math notation="LaTeX">${L_{2}}$ </tex-math></inline-formula>-norm to measure the distance along space dimension and <inline-formula> <tex-math notation="LaTeX">${L_{1}}$ </tex-math></inline-formula>-norm to sum overall data points. The proposed robust graph embedding framework retains the traditional framework’s desirable properties, such as rotational invariance and well geometric structure, and simultaneously suppresses outliers. Based on this framework, we develop a simple and robust feature extraction method, namely <inline-formula> <tex-math notation="LaTeX">${L_{2,1}}$ </tex-math></inline-formula>-norm-based discriminant locality preserving projections (<inline-formula> <tex-math notation="LaTeX">${L_{2,1}}$ </tex-math></inline-formula>-DLPP) and provide an effective iterative algorithm to solve <inline-formula> <tex-math notation="LaTeX">${L_{2,1}}$ </tex-math></inline-formula>-DLPP. Extensive experiments in artificial data and three popular face databases illustrate the effectiveness of our proposed method.

[1]  Feiping Nie,et al.  Robust Distance Metric Learning via Simultaneous L1-Norm Minimization and Maximization , 2014, ICML.

[2]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[4]  Qinghua Zheng,et al.  Avoiding Optimal Mean Robust PCA/2DPCA with Non-greedy ℓ1-Norm Maximization , 2016, IJCAI.

[5]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[6]  Xi-Zhao Wang,et al.  L1-Norm-Based 2DLPP , 2011, 2011 Chinese Control and Decision Conference (CCDC).

[7]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[10]  Jian Yang,et al.  Nuclear Norm-Based 2-DPCA for Extracting Features From Images , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jiashu Zhang,et al.  Discriminant Locality Preserving Projections Based on L1-Norm Maximization , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Dacheng Tao,et al.  Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Angus E. Taylor.  Introduction to functional analysis , 1959 .

[15]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[16]  Qianqian Wang,et al.  Robust 2DPCA and Its Application , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[18]  David J. Kriegman,et al.  Recognition using class specific linear projection , 1997 .

[19]  Xinbo Gao,et al.  Dimensionality Reduction by Integrating Sparse Representation and Fisher Criterion and its Applications , 2015, IEEE Transactions on Image Processing.

[20]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[21]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Yuan Yuan,et al.  Outlier-resisting graph embedding , 2010, Neurocomputing.

[23]  Xuelong Li,et al.  Parameter Free Large Margin Nearest Neighbor for Distance Metric Learning , 2017, AAAI.

[24]  Rong Wang,et al.  Robust 2DPCA With Non-greedy $\ell _{1}$ -Norm Maximization for Image Analysis , 2015, IEEE Transactions on Cybernetics.

[25]  Jing-Yu Yang,et al.  A generalized Foley-Sammon transform based on generalized fisher discriminant criterion and its application to face recognition , 2003, Pattern Recognit. Lett..

[26]  Jingjing Liu,et al.  Enhanced fisher discriminant criterion for image recognition , 2012, Pattern Recognit..

[27]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Jing Wang,et al.  Generalized 2-D Principal Component Analysis by Lp-Norm for Image Analysis , 2016, IEEE Transactions on Cybernetics.

[29]  Hongyuan Zha,et al.  {\it R}$_{\mbox{1}}$-PCA: rotational invariant {\it L}$_{\mbox{1}}$-norm principal component analysis for robust subspace factorization , 2006, ICML 2006.

[30]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[31]  Ling Shao,et al.  Local Feature Discriminant Projection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Feiping Nie,et al.  A Non-Greedy Algorithm for L1-Norm LDA , 2017, IEEE Transactions on Image Processing.

[34]  Shuicheng Yan,et al.  Robust Neighborhood Preserving Projection by Nuclear/L2,1-Norm Regularization for Image Feature Extraction , 2017, IEEE Transactions on Image Processing.

[35]  Yuan-Hai Shao,et al.  Robust L1-norm two-dimensional linear discriminant analysis , 2015, Neural Networks.

[36]  Xiaolong Teng,et al.  Face recognition using discriminant locality preserving projections , 2006, Image Vis. Comput..

[37]  Yuxiao Hu,et al.  Learning a locality preserving subspace for visual recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[38]  Feiping Nie,et al.  Feature Selection via Joint Embedding Learning and Sparse Regression , 2011, IJCAI.

[39]  Dacheng Tao,et al.  Double Shrinking Sparse Dimension Reduction , 2013, IEEE Transactions on Image Processing.

[40]  Zilan Hu,et al.  Comment on: "Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition" , 2008, Pattern Recognit..

[41]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[42]  Haixian Wang,et al.  L1-Norm Kernel Discriminant Analysis Via Bayes Error Bound Optimization for Robust Feature Extraction , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[43]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Jian Yang,et al.  Two-dimensional discriminant transform for face recognition , 2005, Pattern Recognit..

[45]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[46]  Dewen Hu,et al.  Comment on: "Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition" , 2008, Pattern Recognit..

[47]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[48]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[49]  Xuelong Li,et al.  Block Principal Component Analysis With Nongreedy $\ell _{1}$ -Norm Maximization , 2016, IEEE Transactions on Cybernetics.

[50]  Wai Keung Wong,et al.  Joint Tensor Feature Analysis For Visual Object Recognition , 2015, IEEE Transactions on Cybernetics.

[51]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[52]  Xinbo Gao,et al.  Stable Orthogonal Local Discriminant Embedding for Linear Dimensionality Reduction , 2013, IEEE Transactions on Image Processing.

[53]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Xuelong Li,et al.  Low-Rank Preserving Projections , 2016, IEEE Transactions on Cybernetics.

[55]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[56]  Jian Yang,et al.  Ieee Transactions on Image Processing 1 Tensor Discriminant Color Space for Face Recognition , 2022 .

[57]  Qianqian Wang,et al.  Two-Dimensional PCA with F-Norm Minimization , 2017, AAAI.