Feature self-representation based hypergraph unsupervised feature selection via low-rank representation

Dimension reduction methods always catch many attentions, because it could effectively solve the curse of dimensionality problem. In this paper, we propose an unsupervised feature selection method which could efficiently select a subset of informative features from unlabeled data. We integrate the low-rank constraint, hypergraph theory, and the self-representation property of features in a unified framework to conduct unsupervised feature selection. Specifically, we represent each feature by other features to conduct unsupervised feature selection via the feature-level self-representation property. We then embed a low-rank constraint to consider the relations among features. Moreover, a hypergarph regularizer is utilized to consider both the high-order relations and the local structure of the data. This enables the proposed model to take into account both the global structure of the data (via the low-rank constraint) and the local structure of the data (via the hypgergraph regularizer). We use an 2, p-norm regularizer to satisfy the constraints. Therefore, the proposed model is more robust to the previous models due to achieving better feature selection model. Experimental results on benchmark datasets showed that the proposed method effectively selected the most informative features by removing the adverse effect of irrelevant/redundant features, compared to the state-of-the-art methods.

[1]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[2]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[3]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[4]  Chengqi Zhang,et al.  Semi-parametric optimization for missing data imputation , 2007, Applied Intelligence.

[5]  Chengqi Zhang,et al.  Cost-Sensitive Imputing Missing Values with Ordering , 2007, AAAI.

[6]  Dinggang Shen,et al.  Sparse Discriminative Feature Selection for Multi-class Alzheimer's Disease Classification , 2014, MLMI.

[7]  Zili Zhang,et al.  Missing Value Estimation for Mixed-Attribute Data Sets , 2011, IEEE Transactions on Knowledge and Data Engineering.

[8]  Shichao Zhang,et al.  Clustering-based Missing Value Imputation for Data Preprocessing , 2006, 2006 4th IEEE International Conference on Industrial Informatics.

[9]  Xuelong Li,et al.  Learning k for kNN Classification , 2017, ACM Trans. Intell. Syst. Technol..

[10]  Nicu Sebe,et al.  Optimal graph learning with partial tags and multiple features for image and video annotation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Simon Lucey,et al.  Convolutional Sparse Coding for Trajectory Reconstruction , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  GongYihong,et al.  Unsupervised Image Categorization by Hypergraph Partition , 2011 .

[13]  Zhenhua Guo,et al.  A Framework of Joint Graph Embedding and Sparse Regression for Dimensionality Reduction , 2015, IEEE Transactions on Image Processing.

[14]  Shichao Zhang,et al.  Self-representation nearest neighbor search for classification , 2016, Neurocomputing.

[15]  Rongcheng Liu,et al.  An Unsupervised Feature Selection Algorithm: Laplacian Score Combined with Distance-Based Entropy Measure , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[16]  Guang-Hong Yang,et al.  Robust Distributed Fault Estimation for a Network of Dynamical Systems , 2018, IEEE Transactions on Control of Network Systems.

[17]  Jim Jing-Yan Wang,et al.  Semi-supervised local-learning-based feature selection , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[18]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[20]  Feiping Nie,et al.  Unsupervised Feature Selection via Unified Trace Ratio Formulation and K-means Clustering (TRACK) , 2014, ECML/PKDD.

[21]  Xianzhong Long,et al.  Graph Based Semi-Supervised Learning via Structure Preserving Low-Rank Representation , 2015, Neural Processing Letters.

[22]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[23]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Wai Keung Wong,et al.  Optimal Feature Selection for Robust Classification via l2,1-Norms Regularization , 2014, 2014 22nd International Conference on Pattern Recognition.

[25]  Jieping Ye,et al.  Hypergraph spectral learning for multi-label classification , 2008, KDD.

[26]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[27]  Xiaofeng Zhu,et al.  A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis , 2014, NeuroImage.

[28]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[29]  Yukyee Leung,et al.  A Multiple-Filter-Multiple-Wrapper Approach to Gene Selection and Microarray Data Classification , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[30]  C. Ding,et al.  On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions , 2013, KDD.

[31]  Lei Shi,et al.  Robust Spectral Learning for Unsupervised Feature Selection , 2014, 2014 IEEE International Conference on Data Mining.

[33]  M. Wegkamp,et al.  Optimal selection of reduced rank estimators of high-dimensional matrices , 2010, 1004.2995.

[34]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[35]  Parham Moradi,et al.  An unsupervised feature selection algorithm based on ant colony optimization , 2014, Eng. Appl. Artif. Intell..

[36]  Ratna Babu Chinnam,et al.  mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification , 2011, Inf. Sci..

[37]  Li Zhao,et al.  Manifold based fisher method for semi-supervised feature selection , 2013, 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[38]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[39]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[40]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[41]  Shichao Zhang,et al.  Feature selection by combining subspace learning with sparse representation , 2017, Multimedia Systems.

[42]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.