Feature selection based on brain storm optimization for data classification

Abstract Brain storm optimization (BSO) is a new and effective swarm intelligence method inspired by the human brainstorming process. This paper presents a novel BSO-based feature selection technique for data classification. Specifically, the Fuzzy ARTMAP (FAM) model, which is employed as an incremental learning neural network, is combined with BSO, which acts as a feature selection method, to produce the hybrid FAM-BSO model for feature selection and optimization. Firstly, FAM is used to create a number of prototype nodes incrementally. Then, BSO is used to search and select an optimal sub-set of features that is able to produce high accuracy with the minimum number of features. Ten benchmark problems and a real-world case study are employed to evaluate the performance of FAM-BSO. The results are quantified statistically using the bootstrap method with the 95% confidence intervals. The outcome indicates that FAM-BSO is able to produce promising results as compared with those from original FAM and other feature selection methods including particle swarm optimization, genetic algorithm, genetic programming, and ant colony optimization.

[1]  Yuhui Shi,et al.  An Optimization Algorithm Based on Brainstorming Process , 2011, Int. J. Swarm Intell. Res..

[2]  D. Agrafiotis,et al.  Feature selection for structure-activity correlation using binary particle swarms. , 2002, Journal of medicinal chemistry.

[3]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[4]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[5]  Akshya Swain,et al.  A two-dimensional (2-D) learning framework for Particle Swarm based feature selection , 2018, Pattern Recognit..

[6]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[7]  Parham Moradi,et al.  Integration of graph clustering with ant colony optimization for feature selection , 2015, Knowl. Based Syst..

[8]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[10]  Yong Xia,et al.  A tribe competition-based genetic algorithm for feature selection in pattern classification , 2017, Appl. Soft Comput..

[11]  Radu-Emil Precup,et al.  Nature-inspired optimal tuning of input membership functions of Takagi-Sugeno-Kang fuzzy models for Anti-lock Braking Systems , 2015, Appl. Soft Comput..

[12]  Amit Kumar Das,et al.  A new hybrid feature selection approach using feature association map for supervised and unsupervised classification , 2017, Expert Syst. Appl..

[13]  Ioan-Daniel Borlea,et al.  Stable Takagi-Sugeno Fuzzy Control Designed by Optimization , 2017 .

[14]  Xiujuan Lei,et al.  Locating Multiple Optima via Brain Storm Optimization Algorithms , 2018, IEEE Access.

[15]  Julie Wilson,et al.  Novel feature selection method for genetic programming using metabolomic 1H NMR data , 2006 .

[16]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[17]  Yi Mei,et al.  An Efficient Feature Selection Algorithm for Evolving Job Shop Scheduling Rules With Genetic Programming , 2017, IEEE Transactions on Emerging Topics in Computational Intelligence.

[18]  Chee Peng Lim,et al.  A hybrid model of fuzzy ARTMAP and genetic algorithm for data classification and rule extraction , 2016, Expert Syst. Appl..

[19]  Dun-Wei Gong,et al.  Feature selection algorithm based on bare bones particle swarm optimization , 2015, Neurocomputing.

[20]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[21]  Rupa G. Mehta,et al.  Impact of Outlier Removal and Normalization Approach in Modified k-Means Clustering Algorithm , 2011 .

[22]  Parham Moradi,et al.  A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy , 2016, Appl. Soft Comput..

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[25]  Chung-Ho Wang,et al.  Improved particle swarm optimization to minimize periodic preventive maintenance cost for series-parallel systems , 2011, Expert Syst. Appl..

[26]  Chee Peng Lim,et al.  A reinforced fuzzy ARTMAP model for data classification , 2018, International Journal of Machine Learning and Cybernetics.

[27]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[28]  Yuhui Shi,et al.  Brain Storm Optimization Algorithm , 2011, ICSI.

[29]  Cheng-Lung Huang,et al.  A distributed PSO-SVM hybrid system with feature selection and parameter optimization , 2008, Appl. Soft Comput..

[30]  Nikhil R. Pal,et al.  Genetic programming for simultaneous feature selection and classifier design , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Stephen Grossberg,et al.  Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions , 1976, Biological Cybernetics.

[32]  Chee Peng Lim,et al.  A Q-learning-based multi-agent system for data classification , 2017, Appl. Soft Comput..

[33]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[34]  Chang Jui-Fang.  A PERFORMANCE COMPARISON BETWEEN GENETIC ALGORITHMS AND PARTICLE SWARM OPTIMIZATION APPLIED IN CONSTRUCTING EQUITY PORTFOLIOS , 2009 .

[35]  J. Enrique Muñoz Expósito,et al.  Flexible Fuzzy Rule Bases Evolution with Swarm Intelligence for Meta-Scheduling in Grid Computing , 2015, Comput. Informatics.

[36]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[37]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[38]  Mengjie Zhang,et al.  Pareto front feature selection based on artificial bee colony optimization , 2018, Inf. Sci..

[39]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..

[40]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[41]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[42]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[43]  Kun-Huang Chen,et al.  An improved artificial immune recognition system with the opposite sign test for feature selection , 2014, Knowl. Based Syst..

[44]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[45]  Hongbing Ji,et al.  TPPFAM: Use of threshold and posterior probability for category reduction in fuzzy ARTMAP , 2014, Neurocomputing.

[46]  Chee Peng Lim,et al.  A multi-objective evolutionary algorithm-based ensemble optimizer for feature selection and classification with neural network models , 2014, Neurocomputing.

[47]  Siddhartha Bhattacharyya,et al.  A group incremental feature selection for classification using rough set theory based genetic algorithm , 2018, Appl. Soft Comput..

[48]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[49]  Thomas Burwick,et al.  Optimal Algorithmic Complexity of Fuzzy ART , 1998, Neural Processing Letters.

[50]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[51]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..