An improved particle swarm optimization for feature selection

Searching for an optimal feature subset in a high-dimensional feature space is an NP-complete problem; hence, traditional optimization algorithms are inefficient when solving large-scale feature selection problems. Therefore, meta-heuristic algorithms have been extensively adopted to solve the feature selection problem efficiently. This study proposes an improved particle swarm optimization IPSO algorithm using the opposite sign test OST. The test increases population diversity in the PSO mechanism, and avoids local optimal trapping by improving the jump ability of flying particles. Data sets collected from UCI machine learning databases are used to evaluate the effectiveness of the proposed approach. Classification accuracy is employed as a criterion to evaluate classifier performance. Results show that the proposed approach outperforms both genetic algorithms and sequential search algorithms.

[1]  A. Jalilian,et al.  A New Approach for Allocation and Sizing of Multiple Active Power-Line Conditioners , 2010, IEEE Transactions on Power Delivery.

[2]  Jong-Bae Park,et al.  An Improved Particle Swarm Optimization for Nonconvex Economic Dispatch Problems , 2010 .

[3]  Farid Melgani,et al.  Classification of Electrocardiogram Signals With Support Vector Machines and Particle Swarm Optimization , 2008, IEEE Transactions on Information Technology in Biomedicine.

[4]  Gerhard Tutz,et al.  Feature Selection and Weighting by Nearest Neighbor Ensembles , 2009 .

[5]  Cheng-Lung Huang,et al.  A distributed PSO-SVM hybrid system with feature selection and parameter optimization , 2008, Appl. Soft Comput..

[6]  Russell C. Eberhart,et al.  Multiobjective optimization using dynamic neighborhood particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[7]  Mehmet Fatih Tasgetiren,et al.  A particle swarm optimization algorithm for makespan and total flowtime minimization in the permutation flowshop sequencing problem , 2007, Eur. J. Oper. Res..

[8]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[9]  Ya-Ju Fan,et al.  Optimizing feature selection to improve medical diagnosis , 2010, Ann. Oper. Res..

[10]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[11]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[12]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[13]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[14]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[15]  Guo-Chang Gu,et al.  Research on particle swarm optimization: a review , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[16]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[17]  David G. Stork,et al.  Pattern Classification , 1973 .

[18]  T. Blackwell,et al.  Particle swarms and population diversity , 2005, Soft Comput..

[19]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[20]  Surya S. Durbha,et al.  Wrapper-Based Feature Subset Selection for Rapid Image Information Mining , 2010, IEEE Geoscience and Remote Sensing Letters.

[21]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[22]  Paul M. Weaver,et al.  Analysis and benchmarking of meta-heuristic techniques for lay-up optimization , 2010 .

[23]  Inés María Galván,et al.  AMPSO: A New Particle Swarm Method for Nearest Neighborhood Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  TreleaIoan Cristian.  The particle swarm optimization algorithm , 2003 .

[25]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[26]  Wang Yuan-yuan,et al.  Particle Swarm Optimization Algorithm , 2009 .

[27]  Jürgen Branke,et al.  Multiswarms, exclusion, and anti-convergence in dynamic environments , 2006, IEEE Transactions on Evolutionary Computation.

[28]  Voratas Kachitvichyanukul,et al.  A particle swarm optimization for the vehicle routing problem with simultaneous pickup and delivery , 2009, Comput. Oper. Res..

[29]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Q. Henry Wu,et al.  MCPSO: A multi-swarm cooperative particle swarm optimizer , 2007, Appl. Math. Comput..

[31]  Pradipta Kishore Dash,et al.  Fast Tracking of Power Quality Disturbance Signals Using an Optimized Unscented Filter , 2009, IEEE Transactions on Instrumentation and Measurement.

[32]  Juanying Xie,et al.  Using support vector machines with a novel hybrid feature selection method for diagnosis of erythemato-squamous diseases , 2011, Expert Syst. Appl..

[33]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[34]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[35]  David D. Lewis,et al.  Feature Selection and Feature Extraction for Text Categorization , 1992, HLT.

[36]  Yuji Takahata,et al.  Conformational analyses and SAR studies of antispermatogenic hexahydroindenopyridines , 2003 .

[37]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[38]  Ju-Hong Lee,et al.  A Model for k-Nearest Neighbor Query Processing Cost in Multidimensional Data Space , 1999, Inf. Process. Lett..

[39]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[40]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[41]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[42]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[43]  D. Agrafiotis,et al.  Feature selection for structure-activity correlation using binary particle swarms. , 2002, Journal of medicinal chemistry.

[44]  Max A. Little,et al.  Accurate Telemonitoring of Parkinson's Disease Progression by Noninvasive Speech Tests , 2009, IEEE Transactions on Biomedical Engineering.

[45]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[46]  Songbo Tan,et al.  Neighbor-weighted K-nearest neighbor for unbalanced text corpus , 2005, Expert Syst. Appl..

[47]  Feng-Chia Li,et al.  Combination of feature selection approaches with SVM in credit scoring , 2010, Expert Syst. Appl..

[48]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[49]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[50]  Xiaodong Li,et al.  This article has been accepted for inclusion in a future issue. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 Locating and Tracking Multiple Dynamic Optima by a Particle Swarm Model Using Speciation , 2022 .