A cooperative coevolution-based pittsburgh learning classifier system embedded with memetic feature selection

Given that real-world classification tasks always have irrelevant or noisy features which degrade both prediction accuracy and computational efficiency, feature selection is an effective data reduction technique showing promising performance. This paper presents a cooperative coevolution framework to make the feature selection process embedded into the classification model construction within the genetic-based machine learning paradigm. The proposed approach utilizes the divide-and-conquer strategy to manage two populations in parallel, corresponding to the selected feature subsets and the rule sets of classifier respectively, in which a memetic feature selection algorithm is adopted to evolve the feature subset population while a Pittsburgh-style learning classifier system is used to carry out the classifier evolution. These two coevolving populations cooperate with each other regarding the fitness evaluation and the final solution is obtained via collaborations between the best individuals from each population. Empirical results on several benchmark data sets chosen from the UCI repository, together with a non-parametric statistical test, validate that the proposed approach is able to deliver classifiers of better prediction accuracy and higher stability with fewer selected features, compared with the original learning classifier system. In addition, the incorporated feature selection process is shown to help improve the computational efficiency as well.

[1]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[2]  Pablo Moscato,et al.  A Gentle Introduction to Memetic Algorithms , 2003, Handbook of Metaheuristics.

[3]  M.J. Martin-Bautista,et al.  A survey of genetic feature selection in mining issues , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[4]  Jaume Bacardit Peñarroya Pittsburgh genetic-based machine learning in the data mining era: representations, generalization, and run-time , 2004 .

[5]  Daniel R. Tauritz,et al.  A no-free-lunch framework for coevolution , 2008, GECCO '08.

[6]  Peter Funk,et al.  Combined feature selection and similarity modelling in case-based reasoning using hierarchical memetic algorithm , 2010, IEEE Congress on Evolutionary Computation.

[7]  Sean Luke,et al.  Archive-based cooperative coevolutionary algorithms , 2006, GECCO '06.

[8]  R. Paul Wiegand,et al.  Improving Coevolutionary Search for Optimal Multiagent Behaviors , 2003, IJCAI.

[9]  Jason H. Moore,et al.  Learning classifier systems: a complete introduction, review, and roadmap , 2009 .

[10]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[11]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[12]  Francisco Herrera,et al.  IFS-CoCo: Instance and feature selection based on cooperative coevolution with nearest neighbor rule , 2010, Pattern Recognit..

[13]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[14]  David H. Wolpert,et al.  Coevolutionary free lunches , 2005, IEEE Transactions on Evolutionary Computation.

[15]  Ester Bernadó-Mansilla,et al.  Genetic-based machine learning systems are competitive for pattern recognition , 2008, Evol. Intell..

[16]  Abbes Amira,et al.  Feature selection using tabu search for improving the classification rate prostate needle biopsies , 2004, ICPR 2004.

[17]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[18]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[19]  C. C. Lau,et al.  Gene signature selection for cancer prediction using an integrated approach of genetic algorithm and support vector machine , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[20]  Shuo-Yan Chou,et al.  A simulated-annealing-based approach for simultaneous parameter optimization and feature selection of back-propagation networks , 2008, Expert Syst. Appl..

[21]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[22]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[23]  Francisco Herrera,et al.  A First Study on the Use of Coevolutionary Algorithms for Instance and Feature Selection , 2009, HAIS.

[24]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[25]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[26]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[27]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[28]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).