Supervised Feature Selection Algorithm Based on Low-Rank and Manifold Learning

In this paper we show that manifold learning could effectively find the essential dimension of nonlinear high-dimensional data, but it could not use class label information of the data because it is an unsupervised learning method. This paper explores a novel supervised feature selection algorithm based on low-rank and manifold learning. Specifically, we obtain the coefficient matrix according to the relationship between data and class label. Then we combine sparse learning and manifold learning to conduct feature selection. Finally, we use the low-rank representation to further adjust the result of feature selection. Experimental results show that our new method obtains the best results on the four public datasets when compared with six existing methods.

[1]  Yi Yang,et al.  A Convex Formulation for Semi-Supervised Multi-Label Feature Selection , 2014, AAAI.

[2]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Shichao Zhang,et al.  "Missing is useful": missing values in cost-sensitive decision trees , 2005, IEEE Transactions on Knowledge and Data Engineering.

[4]  Shichao Zhang,et al.  Shell-neighbor method and its application in missing data imputation , 2011, Applied Intelligence.

[5]  Nicu Sebe,et al.  Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks , 2013, IEEE Transactions on Multimedia.

[6]  Liang-Tien Chia,et al.  Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[8]  Chengqi Zhang,et al.  POP algorithm: Kernel-based imputation to treat missing values in knowledge discovery from databases , 2009, Expert Syst. Appl..

[9]  Xiaofeng Zhu,et al.  Graph self-representation method for unsupervised feature selection , 2017, Neurocomputing.

[10]  Xuelong Li,et al.  Graph PCA Hashing for Similarity Search , 2017, IEEE Transactions on Multimedia.

[11]  Liang Du,et al.  Unsupervised Feature Selection with Adaptive Structure Learning , 2015, KDD.

[12]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[13]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[14]  Lei Wang,et al.  Efficient Spectral Feature Selection with Minimum Redundancy , 2010, AAAI.

[15]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[16]  Qinghua Hu,et al.  Non-convex regularized self-representation for unsupervised feature selection , 2015, Image Vis. Comput..

[17]  Xiaofeng Zhu,et al.  Missing data imputation by utilizing information within incomplete instances , 2011, J. Syst. Softw..

[18]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[19]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[20]  Chris H. Q. Ding,et al.  Linear Discriminant Analysis: New Formulations and Overfit Analysis , 2011, AAAI.

[21]  Yunming Ye,et al.  MR-NTD: Manifold Regularization Nonnegative Tucker Decomposition for Tensor Data Dimension Reduction and Representation , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[23]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[24]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Jie Tian,et al.  Incorporating MRI structural information into bioluminescence tomography: system, heterogeneous reconstruction and in vivo quantification. , 2014, Biomedical optics express.

[26]  Dinggang Shen,et al.  A Novel Multi-relation Regularization Method for Regression and Classification in AD Diagnosis , 2014, MICCAI.

[27]  Xindong Wu,et al.  Multi-Database Mining , 2003, IEEE Intell. Informatics Bull..