Elitist Binary Wolf Search Algorithm for Heuristic Feature Selection in High-Dimensional Bioinformatics Datasets

Due to the high-dimensional characteristics of dataset, we propose a new method based on the Wolf Search Algorithm (WSA) for optimising the feature selection problem. The proposed approach uses the natural strategy established by Charles Darwin; that is, ‘It is not the strongest of the species that survives, but the most adaptable’. This means that in the evolution of a swarm, the elitists are motivated to quickly obtain more and better resources. The memory function helps the proposed method to avoid repeat searches for the worst position in order to enhance the effectiveness of the search, while the binary strategy simplifies the feature selection problem into a similar problem of function optimisation. Furthermore, the wrapper strategy gathers these strengthened wolves with the classifier of extreme learning machine to find a sub-dataset with a reasonable number of features that offers the maximum correctness of global classification models. The experimental results from the six public high-dimensional bioinformatics datasets tested demonstrate that the proposed method can best some of the conventional feature selection methods up to 29% in classification accuracy, and outperform previous WSAs by up to 99.81% in computational time.

[1]  Moshe Ben-Bassat,et al.  35 Use of distance measures, information measures and error bounds in feature evaluation , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[2]  Mark Michael,et al.  Experimental Study of Information Measure and Inter-Intra Class Distance Ratios on Feature Selection and Orderings , 1973, IEEE Trans. Syst. Man Cybern..

[3]  Simon Fong,et al.  Adaptive swarm cluster-based dynamic multi-objective synthetic minority oversampling technique algorithm for tackling binary imbalanced datasets in biomedical data classification , 2016, BioData Mining.

[4]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[5]  Simon Fong,et al.  Eidetic Wolf Search Algorithm with a global memory structure , 2016, Eur. J. Oper. Res..

[6]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.

[7]  Huan Liu,et al.  A Probabilistic Approach to Feature Selection - A Filter Solution , 1996, ICML.

[8]  María José del Jesús,et al.  Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems , 2001, Inf. Sci..

[9]  Liujuan Cao,et al.  A novel features ranking metric with application to scalable visual and bioinformatics data classification , 2016, Neurocomputing.

[10]  William H. Hsu,et al.  Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning , 2004, Inf. Sci..

[11]  Ying Ju,et al.  Pretata: predicting TATA binding proteins with novel features and dimensionality reduction strategy , 2016, BMC Systems Biology.

[12]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[13]  Amy Jo Kim,et al.  Community Building on the Web: Secret Strategies for Successful Online Communities , 2000 .

[14]  Simon Fong,et al.  Selecting Optimal Feature Set in High-Dimensional Data by Swarm Search , 2013, J. Appl. Math..

[15]  Simon Fong,et al.  Adaptive multi-objective swarm fusion for imbalanced data classification , 2018, Inf. Fusion.

[16]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[17]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Xin-She Yang,et al.  Binary bat algorithm , 2013, Neural Computing and Applications.

[19]  Arthur I. Cohen,et al.  A Branch-and-Bound Algorithm for Unit Commitment , 1983, IEEE Transactions on Power Apparatus and Systems.

[20]  Hans-Peter Kriegel,et al.  The pyramid-technique: towards breaking the curse of dimensionality , 1998, SIGMOD '98.

[21]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[22]  Jinyan Li,et al.  Optimizing SMOTE by Metaheuristics with Neural Network and Decision Tree , 2015, 2015 3rd International Symposium on Computational and Business Intelligence (ISCBI).

[23]  Deniz Erdogmus,et al.  Feature selection in MLPs and SVMs based on maximum output information , 2004, IEEE Transactions on Neural Networks.

[24]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[26]  Kewei Cheng,et al.  Feature Selection , 2016, ACM Comput. Surv..

[27]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[28]  Simon Fong,et al.  Wolf search algorithm with ephemeral memory , 2012, Seventh International Conference on Digital Information Management (ICDIM 2012).

[29]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[30]  L. N. Kanal,et al.  Handbook of Statistics, Vol. 2. Classification, Pattern Recognition and Reduction of Dimensionality. , 1985 .

[31]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[32]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[33]  Simon Fong,et al.  Feature Selection in Life Science Classification: Metaheuristic Swarm Search , 2014, IT Professional.

[34]  Chen Lin,et al.  LibD3C: Ensemble classifiers with a clustering and dynamic selection strategy , 2014, Neurocomputing.

[35]  Ratna Babu Chinnam,et al.  mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification , 2011, Inf. Sci..

[36]  Athanasios V. Vasilakos,et al.  Advances of applying metaheuristics to data mining techniques , 2015 .

[37]  Ke Chen,et al.  Survey of MapReduce frame operation in bioinformatics , 2013, Briefings Bioinform..

[38]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[39]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[40]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[41]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[42]  Simon Fong,et al.  Solving the Under-Fitting Problem for Decision Tree Algorithms by Incremental Swarm Optimization in Rare-Event Healthcare Classification , 2016 .

[43]  Marko Robnik-Sikonja,et al.  An adaptation of Relief for attribute estimation in regression , 1997, ICML.

[44]  Zne-Jung Lee,et al.  Parameter determination of support vector machine and feature selection using simulated annealing approach , 2008, Appl. Soft Comput..