Subspace learning for unsupervised feature selection via adaptive structure learning and rank approximation

Abstract Traditional unsupervised feature selection methods usually construct a fixed similarity matrix. This matrix is sensitive to noise and becomes unreliable, which affects the performance of feature selection. The researches have shown that both the global reconstruction information and local structure information are important for feature selection. To solve the above problem effectively and make use of the global and local information of data simultaneously, a novel algorithm is proposed in this paper, called subspace learning for unsupervised feature selection via adaptive structure learning and rank approximation (SLASR). Specifically, SLASR learns the manifold structure adaptively, thus the preserved local geometric structure can be more accurate and more robust to noise. As a result, the learning of the similarity matrix and the low-dimensional embedding is completed in one step, which improves the effect of feature selection. Meanwhile, SLASR adopts the matrix factorization subspace learning framework. By minimizing the reconstruction error of subspace learning residual matrix, the global reconstruction information of data is preserved. Then, to guarantee more accurate manifold structure of the similarity matrix, a rank constraint is used to constrain the Laplacian matrix. Additionally, the l2,1/2 regularization term is used to constrain the projection matrix to select the most sparse and robust features. Experimental results on twelve benchmark datasets show that SLASR is superior to the six comparison algorithms from the literature.

[1]  Xuelong Li,et al.  Unsupervised Feature Selection with Structured Graph Optimization , 2016, AAAI.

[2]  Jian Yang,et al.  Robust unsupervised feature selection by nonnegative sparse subspace learning , 2019, Neurocomputing.

[3]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[4]  Xiaoli Chen,et al.  Global and intrinsic geometric structure embedding for unsupervised feature selection , 2018, Expert Syst. Appl..

[5]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[6]  Witold Pedrycz,et al.  Subspace learning for unsupervised feature selection via matrix factorization , 2015, Pattern Recognit..

[7]  Ronghua Shang,et al.  Non-Negative Spectral Learning and Sparse Regression-Based Dual-Graph Regularized Feature Selection , 2018, IEEE Transactions on Cybernetics.

[8]  Alexander Rakhlin,et al.  Stability of $K$-Means Clustering , 2006, NIPS.

[9]  Hongtao Lu,et al.  Non-negative and sparse spectral clustering , 2014, Pattern Recognit..

[10]  Jianzhong Wang,et al.  Maximum weight and minimum redundancy: A novel framework for feature subset selection , 2013, Pattern Recognit..

[11]  Henri Luchian,et al.  A unifying criterion for unsupervised clustering and feature selection , 2011, Pattern Recognit..

[12]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[13]  Feiping Nie,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Feature Selection via Joint Embedding Learning and Sparse Regression , 2022 .

[14]  Zhenmin Tang,et al.  Feature extraction using local structure preserving discriminant analysis , 2014, Neurocomputing.

[15]  Pichao Wang,et al.  Robust unsupervised feature selection via dual self-representation and manifold regularization , 2018, Knowl. Based Syst..

[16]  Han Wang,et al.  Unsupervised feature selection via low-rank approximation and structure learning , 2017, Knowl. Based Syst..

[17]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Witold Pedrycz,et al.  Unsupervised feature selection via maximum projection and minimum redundancy , 2015, Knowl. Based Syst..

[19]  Ronghua Shang,et al.  Subspace learning-based graph regularized feature selection , 2016, Knowl. Based Syst..

[20]  Feiping Nie,et al.  Orthogonal vs. uncorrelated least squares discriminant analysis for feature extraction , 2012, Pattern Recognit. Lett..

[21]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[22]  Jian Zhang,et al.  Unsupervised spectral feature selection with l1-norm graph , 2016, Neurocomputing.

[23]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[24]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[25]  Yong Shi,et al.  Feature Selection With $\ell_{2,1-2}$ Regularization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[27]  Qinghua Hu,et al.  Subspace clustering guided unsupervised feature selection , 2017, Pattern Recognit..

[28]  Bo Tang,et al.  Semisupervised Feature Selection Based on Relevance and Redundancy Criteria , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[30]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[31]  Shichao Zhang,et al.  Unsupervised feature selection by combining subspace learning with feature self-representation , 2018, Pattern Recognit. Lett..

[32]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[33]  Zenglin Xu,et al.  Discriminative Semi-Supervised Feature Selection Via Manifold Regularization , 2009, IEEE Transactions on Neural Networks.

[34]  Feiping Nie,et al.  Exploiting Combination Effect for Unsupervised Feature Selection by $\ell_{2,0}$ Norm , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Hongwei Hao,et al.  Selecting feature subset with sparsity and low redundancy for unsupervised learning , 2015, Knowl. Based Syst..

[36]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[37]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[38]  Witold Pedrycz,et al.  Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection , 2015, Pattern Recognit..

[39]  Qinghua Hu,et al.  Co-regularized unsupervised feature selection , 2018, Neurocomputing.

[40]  Jianzhong Wang,et al.  Unsupervised feature selection by regularized matrix factorization , 2018, Neurocomputing.