Unsupervised feature selection by regularized matrix factorization

Abstract Feature selection is an interesting and challenging task in data analysis process. In this paper, a novel algorithm named Regularized Matrix Factorization Feature Selection (RMFFS) is proposed for unsupervised feature selection. Compared with other matrix factorization based feature selection methods, a main advantage of our algorithm is that it takes the correlation among features into consideration. Through introducing an inner product regularization into our algorithm, the features selected by RMFFS would not only well represent the original high-dimensional data, but also contain low redundancy. Moreover, a simple yet efficient iteratively updating algorithm is also developed to solve the proposed RMFFS. Extensive experimental results on nine real world databases demonstrate that our proposed method can achieve better performance than some state-of-the-art unsupervised feature selection methods.

[1]  Jianzhong Wang,et al.  Maximum weight and minimum redundancy: A novel framework for feature subset selection , 2013, Pattern Recognit..

[2]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[3]  Feiping Nie,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Feature Selection via Joint Embedding Learning and Sparse Regression , 2022 .

[4]  Kais Allab,et al.  Constraint Selection-Based Semi-supervised Feature Selection , 2011, 2011 IEEE 11th International Conference on Data Mining.

[5]  Jun Fang,et al.  Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals , 2015, IEEE Trans. Signal Process..

[6]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[7]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[8]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[9]  Witold Pedrycz,et al.  Subspace learning for unsupervised feature selection via matrix factorization , 2015, Pattern Recognit..

[10]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[11]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[12]  W. Rudin.  Principles of mathematical analysis , 1964 .

[13]  Stan Lipovetsky,et al.  Singular value decomposition in additive, multiplicative, and logistic forms , 2005, Pattern Recognit..

[14]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[15]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[17]  Jianzhong Wang,et al.  Label propagation based semi-supervised non-negative matrix factorization for feature extraction , 2015, Neurocomputing.

[18]  Seungjin Choi,et al.  Manifold-respecting discriminant nonnegative matrix factorization , 2011, Pattern Recognit. Lett..

[19]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[20]  Witold Pedrycz,et al.  Local-Structure Adaptive Sparse Subspace Learning: An Iterative Approach to Unsupervised Feature Selection , 2015, ArXiv.

[21]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[22]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[23]  Yunde Jia,et al.  FISHER NON-NEGATIVE MATRIX FACTORIZATION FOR LEARNING LOCAL FEATURES , 2004 .

[24]  Xindong Wu,et al.  Feature selection using hierarchical feature clustering , 2011, CIKM '11.

[25]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[26]  I. Jolliffe.  Principal Component Analysis , 2002 .

[27]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[28]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[29]  Hongwei Hao,et al.  Selecting feature subset with sparsity and low redundancy for unsupervised learning , 2015, Knowl. Based Syst..

[30]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .