Unsupervised feature analysis with sparse adaptive learning

Abstract Unsupervised feature learning has played an important role in machine learning due to its ability to save human labor cost. Since the absence of labels in such scenario, a commonly used approach is to select features according to the similarity matrix derived from the original feature space. However, their similarity matrices suffer from noises and redundant features, with which are frequently confronted in high-dimensional data. In this paper, we propose a novel unsupervised feature selection algorithm. Compared with the previous works, there are mainly two merits of the proposed algorithm: (1) The similarity matrix is adaptively adjusted with a comprehensive strategy to fully utilize the information in the projected data and the original data. (2) To guarantee the clarity of the dramatically learned manifold structure, a non-squared l2-norm based sparsity method is imposed into the objective function. The proposed objective function involves several non-smooth constraints, making it difficult to solve. We also design an efficient iterative algorithm to optimize it. Experimental results demonstrate the effectiveness of our algorithm compared with the state-of-the-art algorithms on several kinds of publicly available datasets.

[1]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[2]  B. Mohar.  THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[3]  Nicu Sebe,et al.  Discriminating Joint Feature Analysis for Multimedia Data Understanding , 2012, IEEE Transactions on Multimedia.

[4]  Yi Yang,et al.  Multi-Class Active Learning by Uncertainty Sampling with Diversity Maximization , 2015, International Journal of Computer Vision.

[5]  Feiping Nie,et al.  Semi-supervised orthogonal discriminant analysis via label propagation , 2009, Pattern Recognit..

[6]  Feiping Nie,et al.  Discriminative Unsupervised Dimensionality Reduction , 2015, IJCAI.

[7]  Yi Yang,et al.  Web and Personal Image Annotation by Mining Label Correlation With Relaxed Visual Graph Embedding , 2012, IEEE Transactions on Image Processing.

[8]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[9]  Feiping Nie,et al.  Unsupervised and semi-supervised learning via ℓ1-norm graph , 2011, 2011 International Conference on Computer Vision.

[10]  Nicu Sebe,et al.  Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks , 2013, IEEE Transactions on Multimedia.

[11]  Qi Mao,et al.  Feature selection for unsupervised learning through local learning , 2015, Pattern Recognit. Lett..

[12]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[13]  Vural Aksakalli,et al.  Feature selection via binary simultaneous perturbation stochastic approximation , 2015, Pattern Recognit. Lett..

[14]  Lei Wang,et al.  Efficient Spectral Feature Selection with Minimum Redundancy , 2010, AAAI.

[15]  Jian Zhang,et al.  Unsupervised spectral feature selection with l1-norm graph , 2016, Neurocomputing.

[16]  Yi Yang,et al.  Image Classification by Cross-Media Active Learning With Privileged Information , 2016, IEEE Transactions on Multimedia.

[17]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[18]  Liang Du,et al.  Unsupervised Feature Selection with Adaptive Structure Learning , 2015, KDD.

[19]  Xiaojun Chang,et al.  Semisupervised Feature Analysis by Mining Correlations Among Multiple Tasks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Feiping Nie,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Feature Selection via Joint Embedding Learning and Sparse Regression , 2022 .

[21]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[22]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[23]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[24]  Edwin R. Hancock,et al.  Unsupervised Feature Selection by Graph Optimization , 2015, ICIAP.

[25]  Xuelong Li,et al.  Unsupervised Feature Selection with Structured Graph Optimization , 2016, AAAI.

[26]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.