Adaptive discriminant analysis for semi-supervised feature selection

Abstract As semi-supervised feature selection is becoming much more popular among researchers, many related methods have been proposed in recent years. However, many of these methods first compute a similarity matrix prior to feature selection, and the matrix is then fixed during the subsequent feature selection process. Clearly, the similarity matrix generated from the original dataset is susceptible to the noise features. In this paper, we propose a novel adaptive discriminant analysis for semi-supervised feature selection, namely, SADA. Instead of computing a similarity matrix first, SADA simultaneously learns an adaptive similarity matrix S and a projection matrix W with an iterative process. Moreover. we introduce the l 2 , p norm to control the sparsity of S by adjusting p. Experimental results show that S will become sparser with the decrease of p. The experimental results for synthetic datasets and eight benchmark datasets demonstrate the superiority of SADA, in comparison with 6 semi-supervised feature selection methods.

[1]  Qi Tian,et al.  Multi-view adaptive semi-supervised feature selection with the self-paced learning , 2020, Signal Process..

[2]  Chen Wang,et al.  Discriminative Semi-Supervised Feature Selection via Rescaled Least Squares Regression-Supplement , 2018, AAAI.

[3]  Michel Verleysen,et al.  A graph Laplacian based approach to semi-supervised feature selection for regression problems , 2013, Neurocomputing.

[4]  Haoliang Yuan,et al.  Two-Dimensional Semi-Supervised Feature Selection , 2020, 2020 10th International Conference on Information Science and Technology (ICIST).

[5]  Yong Shi,et al.  Feature Selection With $\ell_{2,1-2}$ Regularization. , 2018, IEEE transactions on neural networks and learning systems.

[6]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[7]  Fadi Dornaika,et al.  Learning a discriminant graph-based embedding with feature selection for image categorization , 2019, Neural Networks.

[8]  Chih-Fong Tsai,et al.  Ensemble feature selection in high dimension, low sample size datasets: Parallel and serial combination approaches , 2020, Knowl. Based Syst..

[9]  Verónica Bolón-Canedo,et al.  Parallel feature selection for distributed-memory clusters , 2019, Inf. Sci..

[10]  Xiaojun Chen,et al.  Local Adaptive Projection Framework for Feature Selection of Labeled and Unlabeled Data , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Hao Wang,et al.  Online Streaming Feature Selection , 2010, ICML.

[12]  Pichao Wang,et al.  Adaptive Hypergraph Embedded Semi-Supervised Multi-Label Image Annotation , 2019, IEEE Transactions on Multimedia.

[13]  Yvan Saeys,et al.  Robust Feature Selection Using Ensemble Feature Selection Techniques , 2008, ECML/PKDD.

[14]  Huan Liu,et al.  Semi-supervised Feature Selection via Spectral Analysis , 2007, SDM.

[15]  H. Luetkepohl.  The Handbook of Matrices , 1996 .

[16]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.

[17]  Bo Tang,et al.  Semisupervised Feature Selection Based on Relevance and Redundancy Criteria , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Feiping Nie,et al.  Semi-supervised Feature Selection via Rescaled Linear Regression , 2017, IJCAI.

[19]  Liberios Vokorokos,et al.  Ensemble feature selection using election methods and ranker clustering , 2019, Inf. Sci..

[20]  Mohammad Masoud Javidi,et al.  Online streaming feature selection using rough sets , 2016, Int. J. Approx. Reason..

[21]  Philip S. Yu,et al.  Forward Semi-supervised Feature Selection , 2008, PAKDD.

[22]  Ji Zhang,et al.  A factor graph model for unsupervised feature selection , 2019, Inf. Sci..

[23]  Rajavel Ramadoss,et al.  A Parallel Multilevel Feature Selection algorithm for improved cancer classification , 2020, J. Parallel Distributed Comput..

[24]  Qiang Shen,et al.  Feature grouping and selection: A graph-based approach , 2021, Inf. Sci..

[25]  Xin Fan,et al.  Feature selection for imbalanced data based on neighborhood rough sets , 2019, Inf. Sci..

[26]  Zenglin Xu,et al.  Discriminative Semi-Supervised Feature Selection Via Manifold Regularization , 2009, IEEE Transactions on Neural Networks.

[27]  Yi Yang,et al.  Semisupervised Feature Selection via Spline Regression for Video Semantic Recognition , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Samuel H. Huang.  Supervised feature selection: A tutorial , 2015, Artif. Intell. Res..

[29]  Yong Luo,et al.  Vector-Valued Multi-View Semi-Supervsed Learning for Multi-Label Image Classification , 2013, AAAI.

[30]  Yong Shi,et al.  Feature Selection With $\ell_{2,1-2}$ Regularization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Xindong Wu,et al.  Online streaming feature selection using adapted Neighborhood Rough Set , 2019, Inf. Sci..

[32]  Zongben Xu,et al.  Shrinkage Degree in $L_{2}$ -Rescale Boosting for Regression , 2015, IEEE Transactions on Neural Networks and Learning Systems.