A hybrid algorithm for feature subset selection in high-dimensional datasets using FICA and IWSSr algorithm

A hybrid method is proposed for efficient subset selection in high-dimensional datasets. The symmetrical uncertainty (SU) criterion is exploited to weight features in filter phase.In wrapper phase, both fuzzy imperialist competitive algorithm (FICA) and Incremental Wrapper Subset Selection with replacement (IWSSr) in weighted feature space are executed to search and find relevant attributes.The proposed method has been assessed by applying on 10 standard high-dimensional datasets. We compared our proposed algorithm with other five hybrid algorithms (LFS, IWSS, IWSSr, BARS, and Grasp) and two filter methods (FCBF and PCA). The comparison between the results of our method and others confirms that our method has the best accuracy.The average number of attributes selected by proposed algorithm is considerably less than the other methods.The diagrams show low convergence time and low number of iterations with regard to other methods. Feature subset selection is a substantial problem in the field of data classification tasks. The purpose of feature subset selection is a mechanism to find efficient subset retrieved from original datasets to increase both efficiency and accuracy rate and reduce the costs of data classification. Working on high-dimensional datasets with a very large number of predictive attributes while the number of instances is presented in a low volume needs to be employed techniques to select an optimal feature subset. In this paper, a hybrid method is proposed for efficient subset selection in high-dimensional datasets. The proposed algorithm runs filter-wrapper algorithms in two phases. The symmetrical uncertainty (SU) criterion is exploited to weight features in filter phase for discriminating the classes. In wrapper phase, both FICA (fuzzy imperialist competitive algorithm) and IWSSr (Incremental Wrapper Subset Selection with replacement) in weighted feature space are executed to find relevant attributes. The new scheme is successfully applied on 10 standard high-dimensional datasets, especially within the field of biosciences and medicine, where the number of features compared to the number of samples is large, inducing a severe curse of dimensionality problem. The comparison between the results of our method and other algorithms confirms that our method has the most accuracy rate and it is also able to achieve to the efficient compact subset.

[1]  Jianzhong Wang,et al.  Maximum weight and minimum redundancy: A novel framework for feature subset selection , 2013, Pattern Recognit..

[2]  Shigeo Abe,et al.  Modified backward feature selection by cross validation , 2005, ESANN.

[3]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[4]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[5]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[6]  Lakhmi C. Jain,et al.  Feature Selection for Data and Pattern Recognition , 2014, Feature Selection for Data and Pattern Recognition.

[7]  Ron Kohavi,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998 .

[8]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[9]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[10]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[11]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[12]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[13]  Silvia Casado Yusta,et al.  Different metaheuristic strategies to solve the feature selection problem , 2009, Pattern Recognit. Lett..

[14]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[15]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[16]  Vipin Kumar,et al.  Introduction to Data Mining, (First Edition) , 2005 .

[17]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[18]  Yudong D. He,et al.  Gene expression profiling predicts clinical outcome of breast cancer , 2002, Nature.

[19]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[20]  D. Hand,et al.  Idiot's Bayes—Not So Stupid After All? , 2001 .

[21]  Hui-Huang Hsu,et al.  Hybrid feature selection by combining filters and wrappers , 2011, Expert Syst. Appl..

[22]  Jose Miguel Puerta,et al.  Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking , 2012, Knowl. Based Syst..

[23]  Justin C. W. Debuse,et al.  Feature Subset Selection within a Simulated Annealing Data Mining Algorithm , 1997, Journal of Intelligent Information Systems.

[24]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[25]  Jose Miguel Puerta,et al.  Incremental Wrapper-based subset Selection with replacement: An advantageous alternative to sequential forward selection , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[26]  Jesús S. Aguilar-Ruiz,et al.  Incremental wrapper-based gene selection from microarray data for cancer classification , 2006, Pattern Recognit..

[27]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[28]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[29]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[30]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[31]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[32]  Hugues Bersini,et al.  A Survey on Filter Techniques for Feature Selection in Gene Expression Microarray Analysis , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[33]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[34]  Ali Amiri,et al.  FICA: fuzzy imperialist competitive algorithm , 2014, Journal of Zhejiang University SCIENCE C.

[35]  Geoff Holmes,et al.  Benchmarking Attribute Selection Techniques for Discrete Class Data Mining , 2003, IEEE Trans. Knowl. Data Eng..

[36]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[37]  Bhaskar D. Rao,et al.  Backward sequential elimination for sparse vector subset selection , 2001, Signal Process..

[38]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[39]  E. Lughofer,et al.  Enhanced genetic operators design for waveband selection in multivariate calibration based on NIR spectroscopy , 2014 .

[40]  Mauricio G. C. Resende,et al.  Greedy Randomized Adaptive Search Procedures , 1995, J. Glob. Optim..

[41]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[42]  Etienne de Klerk,et al.  Solving Standard Quadratic Optimization Problems via Linear, Semidefinite and Copositive Programming , 2002, J. Glob. Optim..

[43]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[44]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[45]  R. Boggia,et al.  Genetic algorithms as a strategy for feature selection , 1992 .

[46]  Caro Lucas,et al.  Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition , 2007, 2007 IEEE Congress on Evolutionary Computation.

[47]  Jesús S. Aguilar-Ruiz,et al.  Best Agglomerative Ranked Subset for Feature Selection , 2008, FSDM.

[48]  Edwin Lughofer,et al.  On-line incremental feature weighting in evolving fuzzy classifiers , 2011, Fuzzy Sets Syst..

[49]  Jose Miguel Puerta,et al.  A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets , 2011, Pattern Recognit. Lett..

[50]  Geoffrey I. Webb,et al.  Not So Naive Bayes: Aggregating One-Dependence Estimators , 2005, Machine Learning.

[51]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .