An unsupervised feature selection algorithm with feature ranking for maximizing performance of the classifiers

Prediction plays a vital role in decision making. Correct prediction leads to right decision making to save the life, energy, efforts, money and time. The right decision prevents physical and material losses and it is practiced in all the fields including medical, finance, environmental studies, engineering and emerging technologies. Prediction is carried out by a model called classifier. The predictive accuracy of the classifier highly depends on the training datasets utilized for training the classifier. The irrelevant and redundant features of the training dataset reduce the accuracy of the classifier. Hence, the irrelevant and redundant features must be removed from the training dataset through the process known as feature selection. This paper proposes a feature selection algorithm namely unsupervised learning with ranking based feature selection (FSULR). It removes redundant features by clustering and eliminates irrelevant features by statistical measures to select the most significant features from the training dataset. The performance of this proposed algorithm is compared with the other seven feature selection algorithms by well known classifiers namely naive Bayes (NB), instance based (IB1) and tree based J48. Experimental results show that the proposed algorithm yields better prediction accuracy for classifiers.

[1]  Jose Miguel Puerta,et al.  Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking , 2012, Knowl. Based Syst..

[2]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[3]  Salvatore Ruggieri,et al.  Efficient C4.5 , 2002, IEEE Trans. Knowl. Data Eng..

[4]  Sinisa Todorovic,et al.  Local-Learning-Based Feature Selection for High-Dimensional Data Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[6]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[7]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  S. Billings,et al.  Feature Subset Selection and Ranking for Data Dimensionality Reduction , 2007 .

[9]  Shameek Ghosh,et al.  Hybrid biogeography based simultaneous feature selection and MHC class I peptide binding prediction using support vector machines and random forests. , 2013, Journal of immunological methods.

[10]  S. Appavu alias Balamurugan,et al.  A novel feature selection framework for automatic web page classification , 2012, Int. J. Autom. Comput..

[11]  Francisco Herrera,et al.  A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.

[12]  Kemal Polat,et al.  A novel hybrid intelligent method based on C4.5 decision tree classifier and one-against-all approach for multi-class classification problems , 2009, Expert Syst. Appl..

[13]  Thomas Plum,et al.  Efficient C , 1985 .

[14]  Eyke Hüllermeier,et al.  Combining Instance-Based Learning and Logistic Regression for Multilabel Classification , 2009, ECML/PKDD.

[15]  Yogesh R. Shepal.  A Fast Clustering-Based Feature Subset Selection Algorithm for High Dimensional Data , 2014 .

[16]  Harun Uguz,et al.  A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm , 2011, Knowl. Based Syst..

[17]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[18]  Saumyadipta Pyne,et al.  Maximum likelihood inference for mixtures of skew Student-t-normal distributions through practical EM-type algorithms , 2012, Stat. Comput..

[19]  Joshua D. Knowles,et al.  Feature subset selection in unsupervised learning via multiobjective optimization , 2006 .

[20]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Lei Liu,et al.  Feature selection with dynamic mutual information , 2009, Pattern Recognit..

[22]  Chun-Nan Hsu,et al.  Bayesian classification for data from the same unknown class , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[23]  James Bailey,et al.  Comments on supervised feature selection by clustering using conditional mutual information-based distances , 2013, Pattern Recognit..

[24]  Edward Szczerbicki,et al.  PREDICTION BASED ON INTEGRATION OF DECISIONAL DNA AND A FEATURE SELECTION ALGORITHM RELIEF-F , 2013, Cybern. Syst..

[25]  Joshua D. Knowles,et al.  Semi-supervised feature selection via multiobjective optimization , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[26]  Zibin Zheng,et al.  Predicting Quality of Service for Selection by Neighborhood-Based Collaborative Filtering , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[28]  Rashedur M. Rahman,et al.  Using and comparing different decision tree classification techniques for mining ICDDR, B Hospital Surveillance data , 2011, Expert Syst. Appl..

[29]  Ming-Chi Lee,et al.  Using support vector machine with a hybrid feature selection method to the stock trend prediction , 2009, Expert Syst. Appl..

[30]  Mário A. T. Figueiredo,et al.  Efficient feature selection filters for high-dimensional data , 2012, Pattern Recognit. Lett..

[31]  Terrence J. Sejnowski,et al.  ICA Mixture Models for Unsupervised Classification of Non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Xue-wen Chen,et al.  Combating the Small Sample Class Imbalance Problem Using Feature Selection , 2010, IEEE Transactions on Knowledge and Data Engineering.

[33]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[34]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[35]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Huan Liu,et al.  Consistency Based Feature Selection , 2000, PAKDD.

[37]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[38]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[39]  R. Rajaram,et al.  Effective and efficient feature selection for large-scale data using Bayes’ theorem , 2009, Int. J. Autom. Comput..