Jointly Sparse Locality Regression for Image Feature Extraction

This paper proposes a novel method called Jointly Sparse Locality Regression (JSLR) for feature extraction and selection. JSLR utilizes joint <inline-formula><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula>-norm minimization on regularization term, and also introduces the locality to characterize the local geometric structure of the data. There are three main contributions in JSLR for face recognition. Firstly, it eliminates the drawback in ridge regression and Linear Discriminant Analysis (LDA) that when the number of the classes is too small, not enough projections can be obtained for feature extraction. Secondly, by using the local geometric structure as the regularization term, JSLR is able to preserve local information and find an embedding subspace which can detect the most essential data manifold structure. Moreover, since the <inline-formula><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula>-norm based loss function is robust to outliers in data points, JSLR provides the joint sparsity for robust feature selection. The theoretical connections of the proposed method and the previous regression methods are explored and the convergence of the proposed algorithm is also proved. Experimental evaluation on several well-known data sets shows the merits of the proposed method on feature selection and classification.

[1]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[2]  Önsen Toygar,et al.  Selection of optimized features and weights on face-iris fusion using distance images , 2015, Comput. Vis. Image Underst..

[3]  Jian-Bo Yang,et al.  An Effective Feature Selection Method via Mutual Information Estimation , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[5]  Jian Yang,et al.  Nuclear-L1 norm joint regression for face reconstruction and recognition with mixed noise , 2015, Pattern Recognit..

[6]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[7]  Tommy W. S. Chow,et al.  A New Feature Selection Scheme Using a Data Distribution Factor for Unsupervised Nominal Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[9]  Laurence Anthony,et al.  Relevant, irredundant feature selection and noisy example elimination , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Ying Tai,et al.  Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Rabab Kreidieh Ward,et al.  Classification via group sparsity promoting regularization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Huawen Liu,et al.  Regression analysis of locality preserving projections via sparse penalty , 2015, Inf. Sci..

[13]  Jian Yang,et al.  A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[15]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[16]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[17]  Fang Liu,et al.  Unsupervised feature selection based on maximum information and minimum redundancy for hyperspectral images , 2016, Pattern Recognit..

[18]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Feiping Nie,et al.  Discriminant Analysis via Joint Euler Transform and $\ell_{2,1}$ -Norm , 2018, IEEE Transactions on Image Processing.

[20]  Haiping Lu,et al.  Multilinear Principal Component Analysis of Tensor Objects for Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[21]  Jian Yang,et al.  Matrix Variate Distribution-Induced Sparse Representation for Robust Image Classification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Jian Zhang,et al.  Semi-supervised feature selection based on local discriminative information , 2016, Neurocomputing.

[23]  Rui Zhang,et al.  A novel feature selection method considering feature interaction , 2015, Pattern Recognit..

[24]  Michael P. Friedlander,et al.  Theoretical and Empirical Results for Recovery From Multiple Measurements , 2009, IEEE Transactions on Information Theory.

[25]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Changyin Sun,et al.  A regularized least square based discriminative projections for feature extraction , 2016, Neurocomputing.

[27]  Dacheng Tao,et al.  Robust Face Recognition via Multimodal Deep Face Representation , 2015, IEEE Transactions on Multimedia.

[28]  Xiaoou Tang,et al.  A Robust Algorithm for Generalized Orthonormal Discriminant Vectors , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[29]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[30]  Jian Yang,et al.  Robust Image Regression Based on the Extended Matrix Variate Power Exponential Distribution of Dependent Noise , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Yuntao Qian,et al.  A Gabor direct fractional-step LDA algorithm for face recognition , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[32]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[33]  Suman K. Mitra,et al.  On some variants of locality preserving projection , 2016, Neurocomputing.

[34]  Yue Gao,et al.  Exploiting Web Images for Semantic Video Indexing Via Robust Sample-Specific Loss , 2014, IEEE Transactions on Multimedia.

[35]  Bo Tang,et al.  BULDP: Biomimetic Uncorrelated Locality Discriminant Projection for Feature Extraction in Face Recognition , 2018, IEEE Transactions on Image Processing.

[36]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[37]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[38]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[39]  Zhengya Sun,et al.  L0-norm Based Structural Sparse Least Square Regression for Feature Selection , 2015, Pattern Recognit..

[40]  I. Jolliffe,et al.  A Modified Principal Component Technique Based on the LASSO , 2003 .

[41]  Feiping Nie,et al.  Convolutional 2D LDA for Nonlinear Dimensionality Reduction , 2017, IJCAI.

[42]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[43]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[44]  Jian Yang,et al.  Robust nuclear norm regularized regression for face recognition with occlusion , 2015, Pattern Recognit..

[45]  Qiuqi Ruan,et al.  Hessian Semi-Supervised Sparse Feature Selection Based on ${L_{2,1/2}}$ -Matrix Norm , 2015, IEEE Transactions on Multimedia.

[46]  Jian Yang,et al.  Local Structure-Based Image Decomposition for Feature Extraction With Applications to Face Recognition , 2013, IEEE Transactions on Image Processing.

[47]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[48]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[49]  Jinhui Tang,et al.  Unsupervised Feature Selection via Nonnegative Spectral Analysis and Redundancy Control , 2015, IEEE Transactions on Image Processing.

[50]  Jiawei Han,et al.  Spectral Regression: A Unified Approach for Sparse Subspace Learning , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[51]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Jianhua Z. Huang,et al.  Sparse Linear Discriminant Analysis with Applications to High Dimensional Low Sample Size Data , 2009 .

[53]  Feiping Nie,et al.  Discriminative Least Squares Regression for Multiclass Classification and Feature Selection , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Jian Yang,et al.  Weighted sparse coding regularized nonconvex matrix regression for robust face recognition , 2017, Inf. Sci..

[55]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[56]  Rabab Kreidieh Ward,et al.  Synthesis and analysis prior algorithms for joint-sparse recovery , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[58]  Guangwei Gao,et al.  Parameterless reconstructive discriminant analysis for feature extraction , 2016, Neurocomputing.

[59]  Ramachandra Raghavendra,et al.  Designing efficient fusion schemes for multimodal biometric systems using face and palmprint , 2011, Pattern Recognit..

[60]  Ling Ma,et al.  Kernel ridge regression classification , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[61]  Rabab Kreidieh Ward,et al.  Robust Classifiers for Data Reduced via Random Projections , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[62]  Jennie Si,et al.  FREL: A Stable Feature Selection Algorithm , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[63]  Zhiyong Zeng,et al.  Feature Selection Based on Dependency Margin , 2015, IEEE Transactions on Cybernetics.

[64]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[65]  Jian Yang,et al.  Sparse discriminative feature selection , 2015, Pattern Recognit..

[66]  Issam Dagher,et al.  Incremental PCA-LDA algorithm , 2010, 2010 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications.

[67]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[68]  Jiawei Han,et al.  Isometric Projection , 2007, AAAI.

[69]  Nicu Sebe,et al.  Flexible Manifold Learning With Optimal Graph for Image and Video Representation , 2018, IEEE Transactions on Image Processing.

[70]  A. Majumdar,et al.  Fast group sparse classification , 2009, Canadian Journal of Electrical and Computer Engineering.

[71]  Jian Yang,et al.  Nuclear Norm-Based 2-DPCA for Extracting Features From Images , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[72]  Jian Yang,et al.  Regularized Robust Coding for Face Recognition , 2012, IEEE Transactions on Image Processing.

[73]  Xuelong Li,et al.  A General Framework for Auto-Weighted Feature Selection via Global Redundancy Minimization , 2019, IEEE Transactions on Image Processing.

[74]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.