Guided Rule Discovery in XCS for High-Dimensional Classification Problems

XCS is a learning classifier system that combines a reinforcement learning scheme with evolutionary algorithms to evolve a population of classifiers in the form of condition-action rules. In this paper, we investigate the effectiveness of XCS in high-dimensional classification problems where the number of features greatly exceeds the number of data instances --- common characteristics of microarray gene expression classification tasks. We introduce a new guided rule discovery mechanisms for XCS, inspired by feature selection techniques commonly used in machine learning. The extracted feature quality information is used to bias the evolutionary operators. The performance of the proposed model is compared with the standard XCS model and a number of well-known machine learning algorithms using benchmark binary classification tasks and gene expression data sets. Experimental results suggests that the guided rule discovery mechanism is computationally efficient, and promotes the evolution of more accurate solutions. The proposed model performs significantly better than comparative algorithms when tackling high-dimensional classification problems.

[1]  Fang-Xiang Wu,et al.  On Determination of Minimum Sample Size for Discovery of Temporal Gene Expression Patterns , 2006, First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06).

[2]  Jason H. Moore,et al.  Exploiting Expert Knowledge in Genetic Programming for Genome-Wide Genetic Analysis , 2006, PPSN.

[3]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[4]  Francisco Herrera,et al.  Genetics-Based Machine Learning for Rule Induction: State of the Art, Taxonomy, and Comparative Study , 2010, IEEE Transactions on Evolutionary Computation.

[5]  Pier Luca Lanzi,et al.  A Study of the Generalization Capabilities of XCS , 1997, ICGA.

[6]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[7]  Hiroshi Motoda,et al.  Book Review: Computational Methods of Feature Selection , 2007, The IEEE intelligent informatics bulletin.

[8]  E. Lander,et al.  Gene expression correlates of clinical prostate cancer behavior. , 2002, Cancer cell.

[9]  Tim Kovacs,et al.  Advances in Learning Classifier Systems , 2001, Lecture Notes in Computer Science.

[10]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[11]  Concha Bielza,et al.  Machine Learning in Bioinformatics , 2008, Encyclopedia of Database Systems.

[12]  Raymond Chiong,et al.  Novel evolutionary algorithms for supervised classification problems: an experimental study , 2011, Evol. Intell..

[13]  Edmund K. Burke,et al.  Parallel Problem Solving from Nature - PPSN IX: 9th International Conference, Reykjavik, Iceland, September 9-13, 2006, Proceedings , 2006, PPSN.

[14]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[15]  Martin V. Butz,et al.  Automated Global Structure Extraction for Effective Local Building Block Processing in XCS , 2006, Evolutionary Computation.

[16]  Ester Bernadó-Mansilla,et al.  New Crossover Operator for Evolutionary Rule Discovery in XCS , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[17]  Larry Bull,et al.  Learning Classifier Systems , 2002, Annual Conference on Genetic and Evolutionary Computation.

[18]  Luis M. San José-Revuelta,et al.  A Hybrid GA-TS Technique with Dynamic Operators and its Application to Channel Equalization and Fiber Tracking , 2008 .

[19]  Martin V. Butz,et al.  Analysis and Improvement of Fitness Exploitation in XCS: Bounding Models, Tournament Selection, and Bilateral Accuracy , 2003, Evolutionary Computation.

[20]  J. Sudbø,et al.  Gene-expression profiles in hereditary breast cancer. , 2001, The New England journal of medicine.

[21]  Jose Crispin Hernandez Hernandez,et al.  A New Combined Filter-Wrapper Framework for Gene Subset Selection with Specialized Genetic Operators , 2010, MCPR.

[22]  Masoud Nikravesh,et al.  Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing) , 2006 .

[23]  Stewart W. Wilson.  Get Real! XCS with Continuous-Valued Inputs , 1999, Learning Classifier Systems.

[24]  Martin V. Butz,et al.  An Algorithmic Description of XCS , 2000, IWLCS.

[25]  R. Casey,et al.  Advances in Pattern Recognition , 1971 .

[26]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Jaume Bacardit,et al.  Smart crossover operator with multiple parents for a Pittsburgh learning classifier system , 2006, GECCO '06.

[28]  Ester Bernadó-Mansilla,et al.  Genetic-based machine learning systems are competitive for pattern recognition , 2008, Evol. Intell..

[29]  Ester Bernadó-Mansilla,et al.  Analysis and improvement of the genetic discovery component of XCS , 2009, Int. J. Hybrid Intell. Syst..

[30]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.