L1-norm and maximum margin criterion based discriminant locality preserving projections via trace Lasso

Discriminant locality preserving projections based on maximum margin criterion (DLPP/MMC) is a useful feature extraction method since it has shown good performances in pattern recognition. The conventional DLPP/MMC, however, is not robust to noises and outliers since its objective function is based on L2-norm. In this paper, we propose a novel L1-norm and maximum margin criterion based discriminant locality preserving projections via trace Lasso (DLPP/MMC-L1TL). L1-norm rather than L2-norm is used in the formulation of DLPP/MMC-L1TL, which makes it be robust to noises and outliers. Besides, in order to improve the performance of DLPP/MMC-L1TL further, we use trace Lasso to regularize the basis vectors. Trace Lasso, which can balance L1-norm and L2-norm and consider sparsity and correlation of data simultaneously, is a recently proposed norm. An iterative procedure for solving DLPP/MMC-L1TL is also proposed in this paper. The experiment results on some data sets demonstrate the effectiveness of DLPP/MMC-L1TL. We propose a L1-norm and MMC based DLPP method via trace Lasso.Our proposed algorithm can simultaneously consider the sparsity and correlation.We also propose an efficient procedure for solving the proposed method.

[1]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[2]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[3]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[5]  Jing Wang,et al.  2DPCA with L1-norm for simultaneously robust and sparse modelling , 2013, Neural Networks.

[6]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Francis R. Bach,et al.  Structured Sparse Principal Component Analysis , 2009, AISTATS.

[8]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[9]  Jiashu Zhang,et al.  Discriminant Locality Preserving Projections Based on L1-Norm Maximization , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Di Zhang,et al.  Global plus local: A complete framework for feature extraction and recognition , 2014, Pattern Recognit..

[12]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[13]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[15]  Jing Wang,et al.  Robust Face Recognition via Adaptive Sparse Representation , 2014, IEEE Transactions on Cybernetics.

[16]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Ognjen Arandjelovic,et al.  Gradient Edge Map Features for Frontal Face Recognition under Extreme Illumination Changes , 2012, BMVC.

[18]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[19]  Xuelong Li,et al.  L1-Norm-Based 2DPCA , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Jiawei Han,et al.  Isometric Projection , 2007, AAAI.

[21]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Haixian Wang,et al.  L1-Norm Kernel Discriminant Analysis Via Bayes Error Bound Optimization for Robust Feature Extraction , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[23]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[24]  Shuicheng Yan,et al.  Correlation Adaptive Subspace Segmentation by Trace Lasso , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[26]  C J Taylor,et al.  Anatomical statistical models and their role in feature extraction. , 2004, The British journal of radiology.

[27]  Ming Yang,et al.  Large-scale image classification: Fast feature extraction and SVM training , 2011, CVPR 2011.

[28]  Francis R. Bach,et al.  Trace Lasso: a trace norm regularization for correlated designs , 2011, NIPS.

[29]  Zhong Jin,et al.  Face recognition using discriminant locality preserving projections based on maximum margin criterion , 2010, Pattern Recognit..

[30]  Meng Joo Er,et al.  Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Gene H. Golub,et al.  Matrix computations , 1983 .

[32]  Zhongfei Zhang,et al.  Linear discriminant analysis using rotational invariant L1 norm , 2010, Neurocomputing.

[33]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[34]  Jian Yang,et al.  Why can LDA be performed in PCA transformed space? , 2003, Pattern Recognit..

[35]  Ognjen Arandjelovic,et al.  Object Matching Using Boundary Descriptors , 2012, BMVC.

[36]  Ognjen Arandjelovic.  Discriminative extended canonical correlation analysis for pattern set matching , 2013, Machine Learning.

[37]  Xiaolong Teng,et al.  Face recognition using discriminant locality preserving projections , 2006, Image Vis. Comput..

[38]  David Zhang,et al.  A Multiple Maximum Scatter Difference Discriminant Criterion for Facial Feature Extraction , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[40]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[41]  Qin Tang,et al.  L1-Norm-Based Common Spatial Patterns , 2012, IEEE Transactions on Biomedical Engineering.

[42]  Wolfram Burgard,et al.  Point feature extraction on 3D range scans taking into account object boundaries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[43]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.