Unsupervised feature selection by manifold regularized self-representation

Unsupervised feature selection has been proven to be an efficient technique in mitigating the curse of dimensionality. It helps to understand and analyze the prevalent high-dimensional unlabeled data. Recently, the self-similarity property of objects, which assumes that a feature can be represented by the linear combination of its relevant features, has been successfully used in unsupervised feature selection. However, it does not take the geometry structure of the sample space into consideration. In this paper, we propose a novel algorithm termed manifold regularized self-representation(MRSR). To preserve the local spatial structure, we incorporate an effective manifold regularization into the objective function. An iterative reweighted least square (IRLS) algorithm is developed to solve the optimization problem and the convergence is proved. Extensive experimental results on several benchmark datasets validate the effectiveness of the proposed method.

[1]  Feiping Nie,et al.  Early Active Learning via Robust Representation and Structured Sparsity , 2013, IJCAI.

[2]  Huan Liu,et al.  Unsupervised feature selection for linked social media data , 2012, KDD.

[3]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[4]  Xiao Wang,et al.  Unsupervised feature selection via Diversity-induced Self-representation , 2017, Neurocomputing.

[5]  Hongyuan Zha,et al.  {\it R}$_{\mbox{1}}$-PCA: rotational invariant {\it L}$_{\mbox{1}}$-norm principal component analysis for robust subspace factorization , 2006, ICML 2006.

[6]  Jiayu Zhou,et al.  Integrating low-rank and group-sparse structures for robust multi-task learning , 2011, KDD.

[7]  Yi Yang,et al.  Ranking with local regression and global alignment for cross media retrieval , 2009, ACM Multimedia.

[8]  Huan Liu,et al.  Embedded Unsupervised Feature Selection , 2015, AAAI.

[9]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[10]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[11]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[12]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Babak Hassibi,et al.  On the reconstruction of block-sparse signals with an optimal number of measurements , 2009, IEEE Trans. Signal Process..

[14]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[15]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[16]  Xiaofei He,et al.  Laplacian Regularized D-Optimal Design for Active Learning and Its Application to Image Retrieval , 2010, IEEE Transactions on Image Processing.

[17]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[18]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[19]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[20]  Farshad Almasganj,et al.  Using Laplacian eigenmaps latent variable model and manifold learning to improve speech recognition accuracy , 2010, Speech Commun..