Robust Sparse Subspace Learning for unsupervised feature selection

Feature selection is an effective technique for dimensionality reduction to get the most useful information from huge raw data. Many spectral feature selection algorithms have been proposed to address the unsupervised feature selection problem, but most of them fail to pay attention to the noises induced during the feature selection process. In this paper, we not only consider the feature structural learning, but also try to avoid these noises induced during the feature selection process. We utilize the feature structural learning to select the discriminant features and use the robust methods to make selected features more reliable. Furthermore, we propose a new unsupervised feature selection algorithm, namely Robust Sparse Subspace Learning Feature Selection(RSS). And we employ a coordinate descendent algorithm to solve the RSS formulation. Experiments are conducted on several popular datasets to validate the effectiveness of our proposed algorithm and results show that this RSS algorithm achieves better results than traditional feature selection algorithms in most cases.

[1]  Huan Liu,et al.  Feature Selection for Clustering: A Review , 2018, Data Clustering: Algorithms and Applications.

[2]  Lei Shi,et al.  Robust Spectral Learning for Unsupervised Feature Selection , 2014, 2014 IEEE International Conference on Data Mining.

[3]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[4]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[6]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[7]  Jian Pei,et al.  Editorial [State of the Transactions] , 2014, IEEE Trans. Knowl. Data Eng..

[8]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[9]  Lior Wolf,et al.  Feature selection for unsupervised and supervised inference: the emergence of sparsity in a weighted-based approach , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Tong Zhang,et al.  A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , 2005, J. Mach. Learn. Res..

[12]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[13]  Daoqiang Zhang,et al.  Constraint Score: A new filter method for feature selection with pairwise constraints , 2008, Pattern Recognit..

[14]  Jun Sun,et al.  Clustering Via Local Regression , 2008, ECML/PKDD.

[15]  Jing Liu,et al.  Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection , 2014, IEEE Transactions on Knowledge and Data Engineering.

[16]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[17]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[18]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[19]  Nicu Sebe,et al.  Web Image Annotation Via Subspace-Sparsity Collaborated Feature Selection , 2012, IEEE Transactions on Multimedia.

[20]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[21]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[22]  Yiming Yang,et al.  A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[23]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[24]  Yong Luo,et al.  Group Sparse Multiview Patch Alignment Framework With View Consistency for Image Classification , 2014, IEEE Transactions on Image Processing.