Unsupervised feature extraction by low-rank and sparsity preserving embedding

Manifold based feature extraction has been proved to be an effective technique in dealing with the unsupervised classification tasks. However, most of the existing works cannot guarantee the global optimum of the learned projection, and they are sensitive to different noises. In addition, many methods cannot catch the discriminative information as much as possible since they only exploit the local structure of data while ignoring the global structure. To address the above problems, this paper proposes a novel graph based feature extraction method named low-rank and sparsity preserving embedding (LRSPE) for unsupervised learning. LRSPE attempts to simultaneously learn the graph and projection in a framework so that the global optimal projection can be obtained. Moreover, LRSPE exploits both global and local information of data for projection learning by imposing the low-rank and sparse constraints on the graph, which promotes the method to obtain a better performance. Importantly, LRSPE is more robust to noise by imposing the l2,1 sparsity norm on the reconstruction errors. Experimental results on both clean and noisy datasets prove that the proposed method can significantly improve classification accuracy and it is robust to different noises in comparison with the state-of-the-art methods.

[1]  Xuelong Li,et al.  Low-Rank Preserving Projections , 2016, IEEE Transactions on Cybernetics.

[2]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Wai Keung Wong,et al.  Robust Discriminant Regression for Feature Extraction , 2018, IEEE Transactions on Cybernetics.

[4]  Yong Zhao,et al.  An Adaptive Background Modeling Method for Foreground Segmentation , 2017, IEEE Transactions on Intelligent Transportation Systems.

[5]  Zuoyong Li,et al.  Inter-class sparsity based discriminative least square regression , 2018, Neural Networks.

[6]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[7]  David Zhang,et al.  Feature Extraction Methods for Palmprint Recognition: A Survey and Evaluation , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[8]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[9]  Jian Yang,et al.  Two-dimensional discriminant transform for face recognition , 2005, Pattern Recognit..

[10]  Lunke Fei,et al.  Robust Sparse Linear Discriminant Analysis , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[12]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[13]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[14]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[15]  Feiping Nie,et al.  Improved MinMax Cut Graph Clustering with Nonnegative Relaxation , 2010, ECML/PKDD.

[16]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[17]  Zhenyu He,et al.  Deep convolutional neural networks for thermal infrared object tracking , 2017, Knowl. Based Syst..

[18]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[20]  Yong Xu,et al.  Locality and similarity preserving embedding for feature selection , 2014, Neurocomputing.

[21]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[22]  Henggui Zhang,et al.  Multi-Views Fusion CNN for Left Ventricular Volumes Estimation on Cardiac MR Images , 2018, IEEE Transactions on Biomedical Engineering.

[23]  David Zhang,et al.  A method for speeding up feature extraction based on KPCA , 2007, Neurocomputing.

[24]  Wai Keung Wong,et al.  Low-Rank Embedding for Robust Image Feature Extraction , 2017, IEEE Transactions on Image Processing.

[25]  Feiping Nie,et al.  Orthogonal locality minimizing globality maximizing projections for feature extraction , 2009 .

[26]  Xuelong Li,et al.  Low-Rank 2-D Neighborhood Preserving Projection for Enhanced Robust Image Representation , 2019, IEEE Transactions on Cybernetics.

[27]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[28]  Feiping Nie,et al.  Self-Weighted Supervised Discriminative Feature Selection , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Yuedong Yao,et al.  Performance analysis of superheated steam injection for heavy oil recovery and modeling of wellbore heat efficiency , 2017 .

[30]  Feiping Nie,et al.  Orthogonal least squares regression for feature extraction , 2016, Neurocomputing.

[31]  Yuedong Yao,et al.  A numerical approach for obtaining type curves of superheated multi-component thermal fluid flow in concentric dual-tubing wells , 2017 .

[32]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[33]  Yong Wang,et al.  Matrix exponential based discriminant locality preserving projections for feature extraction , 2018, Neural Networks.

[34]  Xiao Ma,et al.  Visual object tracking via coefficients constrained exclusive group LASSO , 2018, Machine Vision and Applications.

[35]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[36]  Xuelong Li,et al.  Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation , 2016, IEEE Transactions on Image Processing.

[37]  Zhenyu He,et al.  Joint sparse principal component analysis , 2017, Pattern Recognit..

[38]  Tieniu Tan,et al.  Feature Selection Based on Structured Sparsity: A Comprehensive Study , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[40]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[41]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Deyu Meng,et al.  Improve robustness of sparse PCA by L1-norm maximization , 2012, Pattern Recognit..

[43]  Feiping Nie,et al.  Flexible Shift-Invariant Locality and Globality Preserving Projections , 2014, ECML/PKDD.

[44]  David Zhang,et al.  Visual Classification With Multikernel Shared Gaussian Process Latent Variable Model , 2019, IEEE Transactions on Cybernetics.

[45]  Zuoyong Li,et al.  Background modelling using discriminative motion representation , 2017, IET Comput. Vis..

[46]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[47]  Jianhua Z. Huang,et al.  Sparse Linear Discriminant Analysis with Applications to High Dimensional Low Sample Size Data , 2009 .

[48]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[49]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[50]  Tieniu Tan,et al.  Demographic Analysis from Biometric Data: Achievements, Challenges, and New Frontiers , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Xiaoqiang Lu,et al.  Person Reidentification Based on Elastic Projections , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[52]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[54]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[55]  Yuedong Yao,et al.  The heat and mass transfer characteristics of superheated steam coupled with non-condensing gases in horizontal wells with multi-point injection technique , 2018 .

[56]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[57]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[58]  Jian Yang,et al.  Rotational Invariant Dimensionality Reduction Algorithms , 2017, IEEE Transactions on Cybernetics.

[59]  Lunke Fei,et al.  Low-Rank Preserving Projection Via Graph Regularized Reconstruction , 2019, IEEE Transactions on Cybernetics.

[60]  Xuelong Li,et al.  Locality Adaptive Discriminant Analysis for Spectral–Spatial Classification of Hyperspectral Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[61]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[62]  Chunwei Tian,et al.  Low-rank representation with adaptive graph regularization , 2018, Neural Networks.

[63]  Xuelong Li,et al.  Structurally Incoherent Low-Rank Nonnegative Matrix Factorization for Image Classification , 2018, IEEE Transactions on Image Processing.

[64]  Jian Yang,et al.  Adaptive weighted nonnegative low-rank representation , 2018, Pattern Recognit..