Joint sparse representation and locality preserving projection for feature extraction

Traditional graph-based feature extraction methods use two separated procedures, i.e., graph learning and projection learning to perform feature extraction. They make the feature extraction result highly dependent on the quality of the initial fixed graph, while the graph may not be the optimal one for feature extraction. In this paper, we propose a novel unsupervised feature extraction method, i.e., joint sparse representation and locality preserving projection (JSRLPP), in which the graph construction and feature extraction are simultaneously carried out. Specifically, we adaptively learn the similarity matrix by sparse representation, and at the same time, learn the projection matrix by preserving local structure. Compared with traditional feature extraction methods, our approach unifies graph learning and projection learning to a common framework, thus learns a more suitable graph for feature extraction. Experiments on several public image data sets demonstrate the effectiveness of our proposed algorithm.

[1]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[2]  William Zhu,et al.  Multi-label feature selection via feature manifold learning and sparsity regularization , 2018, Int. J. Mach. Learn. Cybern..

[3]  Jian Yang,et al.  Kernel ICA: An alternative formulation and its application to face recognition , 2005, Pattern Recognit..

[4]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Feiping Nie,et al.  Orthogonal locality minimizing globality maximizing projections for feature extraction , 2009 .

[6]  A. Martínez,et al.  The AR face databasae , 1998 .

[7]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[8]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[9]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[11]  Shuicheng Yan,et al.  Semi-supervised Learning by Sparse Representation , 2009, SDM.

[12]  Jian Yang,et al.  Rotational Invariant Dimensionality Reduction Algorithms , 2017, IEEE Transactions on Cybernetics.

[13]  Jian Yang,et al.  Approximate Orthogonal Sparse Embedding for Dimensionality Reduction , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Joseph F. Murray,et al.  Visual Recognition and Inference Using Dynamic Overcomplete Sparse Learning , 2007, Neural Computation.

[15]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[16]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[17]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Wai Keung Wong,et al.  Low-Rank Embedding for Robust Image Feature Extraction , 2017, IEEE Transactions on Image Processing.

[19]  Zizhuang Wang,et al.  Temporal-related Convolutional-Restricted-Boltzmann-Machine capable of learning relational order via reinforcement learning procedure? , 2017, ArXiv.

[20]  Xuan Wang,et al.  Adaptive semi-supervised dimensionality reduction based on pairwise constraints weighting and graph optimizing , 2015, International Journal of Machine Learning and Cybernetics.

[21]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[22]  Xuelong Li,et al.  Unsupervised Large Graph Embedding , 2017, AAAI.

[23]  Yu Xue,et al.  Unsupervised feature selection based on self-representation sparse regression and local similarity preserving , 2017, International Journal of Machine Learning and Cybernetics.

[24]  Feiping Nie,et al.  Flexible Shift-Invariant Locality and Globality Preserving Projections , 2014, ECML/PKDD.

[25]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[26]  Jian Yang,et al.  Adaptive weighted nonnegative low-rank representation , 2018, Pattern Recognit..

[27]  Marian Stewart Bartlett,et al.  Independent component representations for face recognition , 1998, Electronic Imaging.

[28]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[29]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[30]  Xuelong Li,et al.  Orthogonal self-guided similarity preserving projection for classification and clustering , 2017, Neural Networks.

[31]  Nenghai Yu,et al.  Neighborhood Preserving Projections (NPP): A Novel Linear Dimension Reduction Method , 2005, ICIC.

[32]  Lunke Fei,et al.  Robust Sparse Linear Discriminant Analysis , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[35]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[36]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Xuelong Li,et al.  Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation , 2016, IEEE Transactions on Image Processing.

[39]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[40]  Yong Xu,et al.  Locality and similarity preserving embedding for feature selection , 2014, Neurocomputing.

[41]  Jeng-Shyang Pan,et al.  From the idea of "sparse representation" to a representation-based transformation method for feature extraction , 2013, Neurocomputing.

[42]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[43]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[44]  Zhong Jin,et al.  Locality Preserving Projections Based on L1 Graph , 2010, 2010 Chinese Conference on Pattern Recognition (CCPR).

[45]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[46]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[47]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .