Exploring autoencoders for unsupervised feature selection

Feature selection plays an important role in pattern classification. It is especially an important preprocessing task when there are large number of features in comparison to number of patterns as is the case with gene expression data. A new unsupervised feature selection method has been evolved using autoencoders since autoencoders have the capacity to learn the input features without class information. In order to prevent the autoencoder from overtraining, masking has been used and the reconstruction error of masked input features has been used to compute feature weights in moving average manner. A new aggregation function for autoencoder has also been introduced by incorporating correlation between input features to remove the redundancy in selected features set. Comparative performance evaluation on benchmark image and gene expression datasets shows that the proposed method outperforms other unsupervised feature selection methods.

[1]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[2]  Mário A. T. Figueiredo,et al.  Unsupervised feature selection for sparse data , 2011, ESANN.

[3]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[4]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[5]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[6]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[7]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[8]  Mário A. T. Figueiredo,et al.  Efficient unsupervised feature selection for sparse data , 2011, 2011 IEEE EUROCON - International Conference on Computer as a Tool.

[9]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[10]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[11]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[12]  Huan Liu,et al.  Semi-supervised Feature Selection via Spectral Analysis , 2007, SDM.

[13]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.

[14]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[15]  Geoffrey J McLachlan,et al.  Selection bias in gene extraction on the basis of microarray gene-expression data , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Lei Wang,et al.  Efficient Spectral Feature Selection with Minimum Redundancy , 2010, AAAI.

[17]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[18]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.