Graph Regularized Feature Selection with Data Reconstruction

Feature selection is a challenging problem for high dimensional data processing, which arises in many real applications such as data mining, information retrieval, and pattern recognition. In this paper, we study the problem of unsupervised feature selection. The problem is challenging due to the lack of label information to guide feature selection. We formulate the problem of unsupervised feature selection from the viewpoint of graph regularized data reconstruction. The underlying idea is that the selected features not only preserve the local structure of the original data space via graph regularization, but also approximately reconstruct each data point via linear combination. Therefore, the graph regularized data reconstruction error becomes a natural criterion for measuring the quality of the selected features. By minimizing the reconstruction error, we are able to select the features that best preserve both the similarity and discriminant information in the original data. We then develop an efficient gradient algorithm to solve the corresponding optimization problem. We evaluate the performance of our proposed algorithm on text clustering. The extensive experiments demonstrate the effectiveness of our proposed approach.

[1]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[2]  Jian Pei,et al.  How Can I Index My Thousands of Photos Effectively and Automatically? An Unsupervised Feature Selection Approach , 2014, SDM.

[3]  Xuelong Li,et al.  General Averaged Divergence Analysis , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[4]  Wei Liu,et al.  Transductive Component Analysis , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[5]  Jing Liu,et al.  Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection , 2014, IEEE Transactions on Knowledge and Data Engineering.

[6]  Huan Liu,et al.  Unsupervised feature selection for linked social media data , 2012, KDD.

[7]  Huan Liu,et al.  Discriminant Analysis for Unsupervised Feature Selection , 2014, SDM.

[8]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[9]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[10]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[11]  Huan Liu,et al.  Semi-supervised Feature Selection via Spectral Analysis , 2007, SDM.

[12]  András Frank,et al.  On Kuhn's Hungarian Method—A tribute from Hungary , 2005 .

[13]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[14]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[15]  Huan Liu,et al.  Unsupervised Feature Selection for Multi-View Data in Social Media , 2013, SDM.

[16]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[17]  Wei-Ying Ma,et al.  An Evaluation on Feature Selection for Text Clustering , 2003, ICML.

[18]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[19]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[20]  Feiping Nie,et al.  Unsupervised maximum margin feature selection via L2,1-norm minimization , 2012, Neural Computing and Applications.

[21]  Lei Wang,et al.  Efficient Spectral Feature Selection with Minimum Redundancy , 2010, AAAI.

[22]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[23]  Rong Jin,et al.  Online Feature Selection and Its Applications , 2014, IEEE Transactions on Knowledge and Data Engineering.

[24]  Carla E. Brodley,et al.  Feature Subset Selection and Order Identification for Unsupervised Learning , 2000, ICML.

[25]  Philip S. Yu,et al.  Dual active feature and sample selection for graph classification , 2011, KDD.

[26]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[27]  Jinbo Bi,et al.  Active learning via transductive experimental design , 2006, ICML.

[28]  Ying Cui,et al.  Convex Principal Feature Selection , 2010, SDM.

[29]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Dawei Song,et al.  Beyond Redundancies: A Metric-Invariant Method for Unsupervised Feature Selection , 2010, IEEE Transactions on Knowledge and Data Engineering.

[31]  Fei Wang,et al.  FeaFiner: biomarker identification from medical data through feature generalization and selection , 2013, KDD.

[32]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[33]  Qinbao Song,et al.  A Fast Clustering-Based Feature Subset Selection Algorithm for High-Dimensional Data , 2013, IEEE Transactions on Knowledge and Data Engineering.

[34]  David G. Stork,et al.  Pattern Classification , 1973 .

[35]  Lior Wolf,et al.  Feature selection for unsupervised and supervised inference: the emergence of sparsity in a weighted-based approach , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[36]  Nizar Bouguila,et al.  Unsupervised Hybrid Feature Extraction Selection for High-Dimensional Non-Gaussian Data Clustering with Variational Inference , 2013, IEEE Transactions on Knowledge and Data Engineering.

[37]  Mohamed S. Kamel,et al.  An Efficient Greedy Method for Unsupervised Feature Selection , 2011, 2011 IEEE 11th International Conference on Data Mining.

[38]  Charles A. Micchelli,et al.  Maximum entropy and maximum likelihood criteria for feature selection from multivariate data , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[39]  Hujun Bao,et al.  A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[41]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[42]  Nizar Bouguila,et al.  Unsupervised Feature Selection for Accurate Recommendation of High-Dimensional Image Data , 2007, NIPS.

[43]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[44]  Xuelong Li,et al.  Deterministic Column-Based Matrix Decomposition , 2010, IEEE Transactions on Knowledge and Data Engineering.

[45]  Haim Schweitzer,et al.  Pass-efficient unsupervised feature selection , 2013, NIPS.

[46]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[47]  Jiye Liang,et al.  Ieee Transactions on Knowledge and Data Engineering 1 a Group Incremental Approach to Feature Selection Applying Rough Set Technique , 2022 .

[48]  Xuan Li,et al.  Local and Global Discriminative Learning for Unsupervised Feature Selection , 2013, 2013 IEEE 13th International Conference on Data Mining.

[49]  Jieping Ye,et al.  Feature grouping and selection over an undirected graph , 2012, KDD.

[50]  Michael I. Jordan,et al.  A Unified Probabilistic Model for Global and Local Unsupervised Feature Selection , 2011, ICML.

[51]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Fei Wang,et al.  Unsupervised Maximum Margin Feature Selection with manifold regularization , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Huan Liu,et al.  CoSelect: Feature Selection with Instance Selection for Social Media Data , 2013, SDM.

[54]  Feiping Nie,et al.  Feature Selection at the Discrete Limit , 2014, AAAI.

[55]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.