XCS and the Monk's Problems

It has been known for some time that Learning Classifier Systems (LCS) have potential for application as Data Mining tools. Parodi and Bonelli applied the Boole LCS to the Lymphography data set and reported 82% classification rates. More recent work, such as GA-Miner has sought to extend the application of the GA-based classification system to larger commercial data sets, introducing more complex attribute encoding techniques, static niching, and hybrid genetic operators in order to address the problems presented by large search spaces. Despite these results, the traditional LCS formulation has shown itself to be unreliable in the formation of accurate optimal generalisations, which are vital for the reduction of results to a human readable form. XCS has been shown to be capable of generating a complete and optimally accurate mapping of a test environment and therefore presents a new opportunity for the application of Learning Classifier Systems to the classification task in Data Mining. As part of a continuing research effort this paper presents some first results in the application of XCS to a particular Data Mining task. It demonstrates that XCS is able to produce a classification performance and rule set which exceeds the performance of most current Machine Learning techniques when applied to the Monk's problems.

[1]  G. V. Kass.  An Exploratory Technique for Investigating Large Quantities of Categorical Data , 1980 .

[2]  Stewart W. Wilson.  Knowledge Growth in an Artificial Animal , 1985, ICGA.

[3]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[4]  Rick L. Riolo,et al.  Bucket Brigade Performance: II. Default Hierarchies , 1987, ICGA.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Stewart W. Wilson.  Bid Competition and Specificity Reconsidered , 1988, Complex Syst..

[7]  Rick L. Riolo,et al.  The Emergence of Default Hierarchies in Learning Classifier Systems , 1989, ICGA.

[8]  Lashon B. Booker,et al.  Triggered Rule Discovery in Classifier Systems , 1989, ICGA.

[9]  C. Watkins.  Learning from delayed rewards , 1989 .

[10]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[11]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[12]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[13]  Kenneth A. De Jong,et al.  Using genetic algorithms for supervised concept learning , 1990, [1990] Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence.

[14]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[15]  Lashon B. Booker,et al.  Representing Attribute-Based Concepts in a Classifier System , 1990, FOGA.

[16]  Zbigniew Michalewicz,et al.  An Experimental Comparison of Binary and Floating Point Representations in Genetic Algorithms , 1991, ICGA.

[17]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[18]  Alexandre Parodi,et al.  The animat and the physician , 1991 .

[19]  Richard J. Enbody,et al.  Further Research on Feature Selection and Classification Using Genetic Algorithms , 1993, ICGA.

[20]  Stephen F. Smith,et al.  Using Coverage as a Model Building Constraint in Learning Classifier Systems , 1994, Evolutionary Computation.

[21]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[22]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[23]  J. M. Mitchell,et al.  Classical statistical methods , 1995 .

[24]  Hongjun Lu,et al.  NeuroRule: A Connectionist Approach to Data Mining , 1995, VLDB.

[25]  Filippo Neri,et al.  Search-Intensive Concept Induction , 1995, Evolutionary Computation.

[26]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[27]  Rafael Molina,et al.  Modern statistical techniques , 1995 .

[28]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[29]  Erik D. Goodman,et al.  Genetic programming for improved data mining: application to the biochemistry of protein interactions , 1996 .

[30]  er SystemsTim KovacsOctober.  Evolving Optimal Populations with XCS Classi , 1996 .

[31]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[32]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[33]  S. Ronald,et al.  Robust encodings in genetic algorithms: a survey of encoding issues , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[34]  Stewart W. Wilson.  Generalization in the XCS Classifier System , 1998 .

[35]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[36]  T. Kovacs.  XCS Classifier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions , 1998 .

[37]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[38]  A. K. Pujari,et al.  Data Mining Techniques , 2006 .