Unsupervised robust Bayesian feature selection

In this paper, we proposed a generative graphical model for unsupervised robust feature selection. The model assumes that the data are independent and identically sampled from a finite mixture of Student-t distribution for dealing with outliers. The Student t-distribution works as the building block for robust clustering and outlier detection. Random variables that represent the features' saliency are included in the model for feature selection. As a result, the model is expected to simultaneously realise unsupervised clustering, feature selection and outlier detection. The inference is carried out by a tree-structured variational Bayes (VB) algorithm. The feature selection capability is realised by estimating the feature saliencies associated with the features. The adoption of full Bayesian treatment in the model realises automatic model selection. Experimental studies showed that the developed algorithm compares favourably against existing unsupervised Bayesian feature selection algorithm in terms of commonly-used internal and external cluster validity indices on controlled experimental settings and benchmark data sets. The controlled experimental study also showed that the developed algorithm is capable of exposing the outliers and finding the optimal number of components (model selection) accurately.

[1]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jiawei Han,et al.  Joint Feature Selection and Subspace Learning , 2011, IJCAI.

[3]  Aristidis Likas,et al.  Bayesian feature and model selection for Gaussian mixture models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Michel Verleysen,et al.  Resampling methods for parameter-free and robust feature selection with mutual information , 2007, Neurocomputing.

[5]  Jane Labadin,et al.  Feature selection based on mutual information , 2015, 2015 9th International Conference on IT in Asia (CITA).

[6]  Richard Nock,et al.  A hybrid filter/wrapper approach of feature selection using information theory , 2002, Pattern Recognit..

[7]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[8]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[9]  Volker Roth,et al.  Feature Selection in Clustering Problems , 2003, NIPS.

[10]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[11]  Quanquan Gu,et al.  Local Learning Regularized Nonnegative Matrix Factorization , 2009, IJCAI.

[12]  Bernhard Schölkopf,et al.  A Local Learning Approach for Clustering , 2006, NIPS.

[13]  A. Ng.  Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[14]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[15]  Huan Liu,et al.  Semi-supervised Feature Selection via Spectral Analysis , 2007, SDM.

[16]  José Manuel Benítez,et al.  Consistency measures for feature selection , 2008, Journal of Intelligent Information Systems.

[17]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[18]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[19]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yiu-ming Cheung,et al.  Feature Selection and Kernel Learning for Local Learning-Based Clustering , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jonathan M. Garibaldi,et al.  Robust Bayesian Clustering for Replicated Gene Expression Data , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[22]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[23]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[24]  Jianyong Sun,et al.  A fast algorithm for robust mixtures in the presence of measurement errors , 2010, IEEE Trans. Neural Networks.

[25]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[26]  Michael E. Houle,et al.  A correlation-based model for unsupervised feature selection , 2007, CIKM '07.

[27]  Shuang-Hong Yang,et al.  Feature Selection by Nonparametric Bayes Error Minimization , 2008, PAKDD.

[28]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[29]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..