Unsupervised feature selection with graph learning via low-rank constraint

Feature selection is one of the most important machine learning procedure, and it has been successfully applied to make a preprocessing before using classification and clustering methods. High-dimensional features often appear in big data, and it’s characters block data processing. So spectral feature selection algorithms have been increasing attention by researchers. However, most feature selection methods, they consider these tasks as two steps, learn similarity matrix from original feature space (may be include redundancy for all features), and then conduct data clustering. Due to these limitations, they do not get good performance on classification and clustering tasks in big data processing applications. To address this problem, we propose an Unsupervised Feature Selection method with graph learning framework, which can reduce the redundancy features influence and utilize a low-rank constraint on the weight matrix simultaneously. More importantly, we design a new objective function to handle this problem. We evaluate our approach by six benchmark datasets. And all empirical classification results show that our new approach outperforms state-of-the-art feature selection approaches.

[1]  Jianfei Cai,et al.  Semi-supervised manifold-embedded hashing with joint feature representation and classifier learning , 2017, Pattern Recognit..

[2]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[3]  Feiping Nie,et al.  Unsupervised Feature Selection via Unified Trace Ratio Formulation and K-means Clustering (TRACK) , 2014, ECML/PKDD.

[4]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[6]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[7]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Exact Top-k Feature Selection via ℓ2,0-Norm Constraint , 2022 .

[8]  Yi Yang,et al.  Co-Regularized Ensemble for Feature Selection , 2013, IJCAI.

[9]  Liang Du,et al.  Unsupervised Feature Selection with Adaptive Structure Learning , 2015, KDD.

[10]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[11]  Shichao Zhang,et al.  The Journal of Systems and Software , 2012 .

[12]  Qinghua Hu,et al.  Subspace clustering guided unsupervised feature selection , 2017, Pattern Recognit..

[13]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[14]  Xuelong Li,et al.  Unsupervised Feature Selection with Structured Graph Optimization , 2016, AAAI.

[15]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[16]  Yi Yang,et al.  A Convex Formulation for Semi-Supervised Multi-Label Feature Selection , 2014, AAAI.

[17]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[18]  Xiaofeng Zhu,et al.  Efficient kNN Classification With Different Numbers of Nearest Neighbors , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Zenglin Xu,et al.  Discriminative Semi-Supervised Feature Selection Via Manifold Regularization , 2009, IEEE Transactions on Neural Networks.

[20]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[21]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[22]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[23]  Xuelong Li,et al.  Learning k for kNN Classification , 2017, ACM Trans. Intell. Syst. Technol..

[24]  Xiaofeng Zhu,et al.  Graph self-representation method for unsupervised feature selection , 2017, Neurocomputing.

[25]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26]  Qinghua Zheng,et al.  An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition , 2018, IEEE Transactions on Cybernetics.

[27]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[28]  Dinggang Shen,et al.  Subspace Regularized Sparse Multitask Learning for Multiclass Neurodegenerative Disease Identification , 2016, IEEE Transactions on Biomedical Engineering.

[29]  C. Ding,et al.  On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions , 2013, KDD.

[30]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[31]  Sinisa Todorovic,et al.  Local-Learning-Based Feature Selection for High-Dimensional Data Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[33]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[34]  Xuelong Li,et al.  Unsupervised Large Graph Embedding , 2017, AAAI.

[35]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[36]  Henryk Wozniakowski,et al.  The curse of dimensionality for numerical integration of smooth functions , 2012, Math. Comput..

[37]  Philip S. Yu,et al.  Semi-supervised feature selection for graph classification , 2010, KDD.

[38]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[39]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[40]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[41]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[42]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[43]  Massimo Fornasier,et al.  Iteratively Re-weighted Least Squares minimization: Proof of faster than linear rate for sparse recovery , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[44]  Zi Huang,et al.  Self-taught dimensionality reduction on the high-dimensional small-sized data , 2013, Pattern Recognit..

[45]  Witold Pedrycz,et al.  Online Feature Transformation Learning for Cross-Domain Object Category Recognition , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Dinggang Shen,et al.  Low-Rank Graph-Regularized Structured Sparse Regression for Identifying Genetic Biomarkers , 2017, IEEE Transactions on Big Data.