Learning Classifier Systems in Data Mining

Just over thirty years after Holland first presented the outline for Learning Classifier System paradigm, the ability of LCS to solve complex real-world problems is becoming clear. In particular, their capability for rule induction in data mining has sparked renewed interest in LCS. This book brings together work by a number of individuals who are demonstrating their good performance in a variety of domains. The first contribution is arranged as follows: Firstly, the main forms of LCS are described in some detail. A number of historical uses of LCS in data mining are then reviewed before an overview of the rest of the volume is presented. The rest of this book describes recent research on the use of LCS in the main areas of machine learning data mining: classification, clustering, time-series and numerical prediction, feature selection, ensembles, and knowledge discovery.

[1]  George G. Robertson,et al.  A tale of two classifier systems , 1988, Machine Learning.

[2]  Larry Bull,et al.  ZCS Redux , 2002, Evolutionary Computation.

[3]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[4]  Geoffrey J. Barton,et al.  Protein sequence alignments: a strategy for the hierarchical analysis of residue conservation , 1993, Comput. Appl. Biosci..

[5]  Stewart W. Wilson.  Classifier Systems and the Animat Problem , 1987, Machine Learning.

[6]  Christopher Bystroff,et al.  Predicting interresidue contacts using templates and pathways , 2003, Proteins.

[7]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[8]  Robert M. MacCallum,et al.  Striped sheets and protein contact prediction , 2004, ISMB/ECCB.

[9]  Larry Bull,et al.  A memetic accuracy-based neural learning classifier system , 2005, 2005 IEEE Congress on Evolutionary Computation.

[10]  Alexandre Parodi,et al.  An Efficient Classifier System and Its Experimental Comparison with Two Representative Learning Methods on Three Medical Domains , 1991, ICGA.

[11]  Edmund K. Burke,et al.  Multimeme Algorithms for Protein Structure Prediction , 2002, PPSN.

[12]  Gabriela Ochoa,et al.  Evolving L-Systems to Capture Protein Structure Native Conformations , 2005, EuroGP.

[13]  K. Nishikawa,et al.  Predicting absolute contact numbers of native protein structure from amino acid sequence , 2004, Proteins.

[14]  William E. Hart,et al.  Robust Proofs of NP-Hardness for Protein Folding: General Lattices and Energy Potentials , 1997, J. Comput. Biol..

[15]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[16]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[17]  Rick L. Riolo,et al.  Modeling Simple Human Category Learning with a Classifier System , 1991, International Conference on Genetic Algorithms.

[18]  Dave Cliff,et al.  Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[19]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[20]  Natalio Krasnogor,et al.  Fuzzy Memes in Multimeme Algorithms: a Fuzzy-Evolutionary Hybrid , 2003 .

[21]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[22]  Sara J. Graves,et al.  Improving performance of an electrical power expert system with genetic algorithms , 1988, IEA/AIE '88.

[23]  David J. Slate,et al.  Letter Recognition Using Holland-Style Adaptive Classifiers , 1991, Machine Learning.

[24]  Rupert G. Miller.  Simultaneous Statistical Inference , 1966 .

[25]  Kenneth A. De Jong,et al.  Using genetic algorithms for concept learning , 1993, Machine Learning.

[26]  Francine Federman,et al.  Information Theory and NEXTPITCH: A Learning Classifier System , 1997, ICGA.

[27]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[28]  Larry Bull,et al.  Genetic Programming for Kernel-Based Learning with Co-evolving Subsets Selection , 2006, PPSN.

[29]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Stephen F. Smith,et al.  Using Coverage as a Model Building Constraint in Learning Classifier Systems , 1994, Evolutionary Computation.

[31]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[32]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[33]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[34]  Jaume Bacardit,et al.  Empirical Evaluation of Ensemble Techniques for a Pittsburgh Learning Classifier System , 2007, IWLCS.

[35]  Francisco Herrera,et al.  Solving Electrical Distribution Problems Using Hybrid Evolutionary Data Analysis Techniques , 2004, Applied Intelligence.

[36]  Manuel Valenzuela-Rendón,et al.  The Fuzzy Classifier System: A Classifier System for Continuously Varying Variables , 1991, ICGA.

[37]  Hideo Matsuda,et al.  PDB-REPRDB: a database of representative protein chains from the Protein Data Bank (PDB) , 2001, Nucleic Acids Res..

[38]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[39]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[40]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[41]  Sean Saxon,et al.  XCS and the Monk's Problems , 1999, Learning Classifier Systems.

[42]  Jacek Blazewicz,et al.  Coordination number prediction using learning classifier systems: performance and interpretability , 2006, GECCO '06.

[43]  Lashon B. Booker,et al.  Triggered Rule Discovery in Classifier Systems , 1989, ICGA.

[44]  Xavier Llorà,et al.  Automated alphabet reduction method with evolutionary algorithms for protein structure prediction , 2007, GECCO '07.

[45]  Gilles Venturini,et al.  SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes based Concepts , 1993, ECML.

[46]  Pierre Baldi,et al.  The Principled Design of Large-Scale Recursive Neural Network Architectures--DAG-RNNs and the Protein Structure Prediction Problem , 2003, J. Mach. Learn. Res..

[47]  Peter Clark,et al.  Induction in Noisy Domains , 1987, EWSL.

[48]  Haruo Abe,et al.  Noninteracting local‐structure model of folding and unfolding transition in globular proteins. II. Application to two‐dimensional lattice proteins , 1981, Biopolymers.

[49]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[50]  Larry Bull,et al.  Accuracy-based Neuro And Neuro-fuzzy Classifier Systems , 2002, GECCO.

[51]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[52]  Filippo Neri,et al.  Search-Intensive Concept Induction , 1995, Evolutionary Computation.

[53]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[54]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[55]  Moshe Sipper,et al.  A fuzzy-genetic approach to breast cancer diagnosis , 1999, Artif. Intell. Medicine.

[56]  Stewart W. Wilson.  Function approximation with a classifier system , 2001 .

[57]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[58]  D T Jones,et al.  Protein secondary structure prediction based on position-specific scoring matrices. , 1999, Journal of molecular biology.

[59]  T. Kovacs.  XCS Classifier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions , 1998 .

[60]  Ellen R. McGrattan,et al.  Money as a medium of exchange in an economy with artificially intelligent agents , 1990 .

[61]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[62]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[63]  Martin V. Butz,et al.  Hyper-ellipsoidal conditions in XCS: rotation, linear approximation, and solution structure , 2006, GECCO '06.

[64]  Jaume Bacardit Peñarroya.  Pittsburgh genetic-based machine learning in the data mining era: representations, generalization, and run-time , 2004 .

[65]  M. Levitt,et al.  A lattice model for protein structure prediction at low resolution. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[66]  William E. Hart,et al.  Protein structure prediction with evolutionary algorithms , 1999 .

[67]  M. Hecht,et al.  Nature disfavors sequences of alternating polar and non-polar amino acids: implications for amyloidogenesis. , 2000, Journal of molecular biology.

[68]  George Karypis,et al.  Prediction of contact maps using support vector machines , 2003, Third IEEE Symposium on Bioinformatics and Bioengineering, 2003. Proceedings..

[69]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[70]  Martin V. Butz,et al.  Speeding-Up Pittsburgh Learning Classifier Systems: Modeling Time and Accuracy , 2004, PPSN.

[71]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[72]  Arthur L. Samuel,et al.  Some studies in machine learning using the game of checkers , 2000, IBM J. Res. Dev..