Hybrid self-inertia weight adaptive particle swarm optimisation with local search using C4.5 decision tree classifier for feature selection problems

ABSTRACT Feature selection is an important task to improve the classifier’s accuracy and to decrease the problem size. A number of methodologies have been presented for feature selection problems using metaheuristic algorithms. In this paper, an improved self-adaptive inertia weight particle swarm optimisation with local search and combined with C4.5 classifiers for feature selection algorithm is proposed. In this proposed algorithm, the gradient base local search with its capacity of helping to explore the feature space and an improved self-adaptive inertia weight particle swarm optimisation with its ability to converge a best global solution in the search space. Experimental results have verified that the SIW-APSO-LS performed well compared with other state of art feature selection techniques on a suit of 16 standard data sets.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[3]  Marko Robnik-Sikonja,et al.  An adaptation of Relief for attribute estimation in regression , 1997, ICML.

[4]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[5]  Carlos Cotta dash,et al.  A study of hybridisation techniques and their application to the design of evolutionary algorithms , 1998 .

[6]  Carlos Cotta.  A Study of Hybridisation Techniques and Their Application to the Design of Evolutionary Algorithms , 1998, AI Commun..

[7]  D. Agrafiotis,et al.  Feature selection for structure-activity correlation using binary particle swarms. , 2002, Journal of medicinal chemistry.

[8]  R. A. Krohling,et al.  Solving numerical equations of hydraulic problems using particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[9]  S. Halgamuge,et al.  A comparison of constraint-handling methods for the application of particle swarm optimization to constrained nonlinear optimization problems , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[10]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[11]  David A. Bell,et al.  A Formalism for Relevance and Its Application in Feature Subset Selection , 2000, Machine Learning.

[12]  Zhong Yan,et al.  Ant Colony Optimization for Feature Selection in Face Recognition , 2004, ICBA.

[13]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[14]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Nikhil R. Pal,et al.  Genetic programming for simultaneous feature selection and classifier design , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  S. Kotsiantis.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[17]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[18]  Xiaoming Xu,et al.  A hybrid genetic algorithm for feature selection wrapper based on mutual information , 2007, Pattern Recognit. Lett..

[19]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[20]  Karim Faez,et al.  An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system , 2008, Appl. Math. Comput..

[21]  Zhijian Wu,et al.  A Hybrid Particle Swarm Optimization Algorithm Based on Space Transformation Search and a Modified Velocity Model , 2009, HPCA.

[22]  Magdalene Marinaki,et al.  Ant colony and particle swarm optimization for financial classification problems , 2009, Expert Syst. Appl..

[23]  Cheng-Lung Huang,et al.  A hybrid SOFM-SVR with a filter-based feature selection for stock market forecasting , 2009, Expert Syst. Appl..

[24]  Ángel Fernando Kuri Morales,et al.  A search space reduction methodology for data mining in large databases , 2009, Eng. Appl. Artif. Intell..

[25]  Jing Zhao,et al.  A Modified Ant Colony Optimization Algorithm for Tumor Marker Gene Selection , 2009, Genom. Proteom. Bioinform..

[26]  Lior Rokach,et al.  A Survey of Feature Selection Techniques , 2009, Encyclopedia of Data Warehousing and Mining.

[27]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[28]  Kazuyuki Murase,et al.  A new local search based hybrid genetic algorithm for feature selection , 2011, Neurocomputing.

[29]  Zhen Liu,et al.  A new feature selection algorithm based on binomial hypothesis testing for spam filtering , 2011, Knowl. Based Syst..

[30]  Li-Yeh Chuang,et al.  Chaotic maps based on binary particle swarm optimization for feature selection , 2011, Appl. Soft Comput..

[31]  Ratna Babu Chinnam,et al.  mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification , 2011, Inf. Sci..

[32]  Sébastien Paris,et al.  Application of global optimization methods to model and feature selection , 2012, Pattern Recognit..

[33]  M. Noel,et al.  A new gradient based particle swarm optimization algorithm for accurate computation of global minimum , 2012, Appl. Soft Comput..

[34]  Xin-She Yang,et al.  BBA: A Binary Bat Algorithm for Feature Selection , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[35]  Mohammad Saniee Abadeh,et al.  Gene selection for cancer tumor detection using a novel memetic algorithm with a multi-view fitness function , 2013, Eng. Appl. Artif. Intell..

[36]  Mohsen Ramezani,et al.  Improve performance of collaborative filtering systems using backward feature selection , 2013, The 5th Conference on Information and Knowledge Technology.

[37]  Yixin Chen,et al.  Efficient ant colony optimization for image feature selection , 2013, Signal Process..

[38]  Yudong Zhang,et al.  Binary PSO with mutation operator for feature selection using decision tree applied to spam detection , 2014, Knowl. Based Syst..

[39]  Samina Khalid,et al.  A survey of feature selection and feature extraction techniques in machine learning , 2014, 2014 Science and Information Conference.

[40]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[41]  Parham Moradi,et al.  Gene selection for microarray data classification using a novel ant colony optimization , 2015, Neurocomputing.

[42]  Parham Moradi,et al.  A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy , 2016, Appl. Soft Comput..

[43]  Minh Le Nguyen,et al.  Feature weighting and SVM parameters optimization based on genetic algorithms for classification problems , 2016, Applied Intelligence.

[44]  Yudong Zhang,et al.  Detection of abnormal MR brains based on wavelet entropy and feature selection , 2016 .

[45]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[46]  Hossein Nezamabadi-pour,et al.  Application of binary quantum-inspired gravitational search algorithm in feature subset selection , 2017, Applied Intelligence.

[47]  Fei Han,et al.  An improved hybrid self-inertia weight adaptive particle swarm optimization algorithm with local search , 2018, Engineering Optimization.

[48]  Yuyan Han,et al.  Efficient multi-objective optimization algorithm for hybrid flow shop scheduling problems with setup energy consumptions , 2018 .

[49]  Fei Han,et al.  An Improved Hybrid Method Combining Gravitational Search Algorithm With Dynamic Multi Swarm Particle Swarm Optimization , 2019, IEEE Access.

[50]  Junqing Li,et al.  Optimal chiller loading by improved artificial fish swarm algorithm for energy saving , 2019, Math. Comput. Simul..

[51]  Peiyong Duan,et al.  An improved artificial bee colony algorithm for addressing distributed flow shop with distance coefficient in a prefabricated system , 2019, Int. J. Prod. Res..

[52]  Jianqiang Yi,et al.  Interval data driven construction of shadowed sets with application to linguistic word modelling , 2020, Inf. Sci..