Noname manuscript No. (will be inserted by the editor) Learning Classifier Systems: A Survey

Learning classifier systems (LCSs) are rule- based systems that automatically build their ruleset. At the origin of Holland’s work, LCSs were seen as a model of the emergence of cognitive abilities thanks to adaptive mechanisms, particularly evolutionary processes. After a renewal of the field more focused on learning, LCSs are now considered as sequential decision problem-solving systems endowed with a generalization property. Indeed, from a Reinforcement Learning point of view, LCSs can be seen as learning systems building a compact representation of their problem thanks to generalization. More recently, LCSs have proved efficient at solving automatic classification tasks. The aim of the present contribution is to describe the state-of- the-art of LCSs, emphasizing recent developments, and focusing more on the sequential decision domain than on automatic classification.

[1]  Tim Kovacs.  Learning classifier systems resources , 2002, Soft Comput..

[2]  Lashon B. Booker,et al.  Do We Really Need to Estimate Rule Utilities in Classifier Systems? , 1999, Learning Classifier Systems.

[3]  Larry Bull,et al.  Design of a Traffic Junction Controller Using Classifier Systems and Fuzzy Logic , 1999, Fuzzy Days.

[4]  Rick L. Riolo,et al.  Lookahead planning and latent learning in a classifier system , 1991 .

[5]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[6]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[7]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Larry Bull,et al.  A Genetic Programming-based Classifier System , 1999, GECCO.

[10]  Olivier Sigaud,et al.  Chi-square Tests Driven Method for Learning the Structure of Factored MDPs , 2006, UAI.

[11]  Olivier Sigaud,et al.  Improving MACS Thanks to a Comparison with 2TBNs , 2004, GECCO.

[12]  Daniele Loiacono,et al.  Classifier prediction based on tile coding , 2006, GECCO '06.

[13]  Larry Bull,et al.  ZCS Redux , 2002, Evolutionary Computation.

[14]  Stewart W. Wilson.  Get Real! XCS with Continuous-Valued Inputs , 1999, Learning Classifier Systems.

[15]  Tim Kovacs,et al.  Applications of Learning Classifier Systems , 2004 .

[16]  Olivier Sigaud,et al.  Learning the structure of Factored Markov Decision Processes in reinforcement learning problems , 2006, ICML.

[17]  Stewart W. Wilson.  Knowledge Growth in an Artificial Animal , 1985, ICGA.

[18]  Martin V. Butz,et al.  Introducing a Genetic Generalization Pressure to the Anticipatory Classifier System - Part 1: Theoretical approach , 2000, GECCO.

[19]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[20]  Marc Schoenauer,et al.  ATNoSFERES revisited , 2005, GECCO '05.

[21]  Olivier Sigaud,et al.  YACS: a new learning classifier system using anticipation , 2002, Soft Comput..

[22]  Olivier Sigaud,et al.  Designing Efficient Exploration with MACS: Modules and Function Approximation , 2003, GECCO.

[23]  Y J Cao,et al.  AN EVOLUTIONARY INTELLIGENT AGENTS APPROACH TO TRAFFIC SIGNALS CONTROL , 2001 .

[24]  M. Puterman,et al.  Modified Policy Iteration Algorithms for Discounted Markov Decision Problems , 1978 .

[25]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[26]  Pier Luca Lanzi,et al.  Learning classifier systems from a reinforcement learning perspective , 2002, Soft Comput..

[27]  Martin V. Butz,et al.  Toward a theory of generalization and learning in XCS , 2004, IEEE Transactions on Evolutionary Computation.

[28]  Christopher Stone,et al.  Towards Learning Classifier Systems for Continuous-Valued Online Environments , 2003, GECCO.

[29]  Martin V. Butz,et al.  Investigating Generalization in the Anticipatory Classifier System , 2000, PPSN.

[30]  Pier Luca Lanzi,et al.  A Roadmap to the Last Decade of Learning Classifier System Research , 1999, Learning Classifier Systems.

[31]  Stewart W. Wilson.  Classifier Systems for Continuous Payoff Environments , 2004, GECCO.

[32]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[33]  Jaume Bacardit,et al.  Evolving Multiple Discretizations with Adaptive Intervals for a Pittsburgh Rule-Based Learning Classifier System , 2003, GECCO.

[34]  Gabriel Robert.  MHiCS, une architecture de sélection de l'action motivationnelle et hiérarchique à systèmes de classeurs pour personnages non joueurs adaptatifs , 2005 .

[35]  Riccardo Poli,et al.  Schema Theory for Genetic Programming with One-Point Crossover and Point Mutation , 1997, Evolutionary Computation.

[36]  Larry Bull,et al.  On ZCS in Multi-agent Environments , 1998, PPSN.

[37]  Larry Bull,et al.  Foundations of Learning Classifier Systems , 2005 .

[38]  Martin V. Butz,et al.  Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems , 2005, IEEE Transactions on Evolutionary Computation.

[39]  Martin V. Butz,et al.  Tournament Selection: Stable Fitness Pressure in XCS , 2003, GECCO.

[40]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[41]  Stewart W. Wilson,et al.  Learning Classifier Systems, From Foundations to Applications , 2000 .

[42]  Martin V. Butz,et al.  Automated Global Structure Extraction for Effective Local Building Block Processing in XCS , 2006, Evolutionary Computation.

[43]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[44]  Craig Boutilier,et al.  Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[45]  Larry Bull,et al.  On using ZCS in a Simulated Continuous Double-Auction Market , 1999, GECCO.

[46]  J. F. Herbart.  Psychologie als Wissenschaft : neu gegründet auf Erfahrung, Metaphysik und Mathematik , 1824 .

[47]  Riccardo Poli,et al.  Genetic and Evolutionary Computation – GECCO 2004 , 2004, Lecture Notes in Computer Science.

[48]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[49]  Xavier Llorà,et al.  Coevolving Different Knowledge Representations With Fine-grained Parallel Learning Classifier Systems , 2002, GECCO.

[50]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[51]  Martin V. Butz,et al.  An Algorithmic Description of ACS2 , 2001, International Workshop on Learning Classifier Systems.

[52]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[53]  Martin V. Butz,et al.  Generalized State Values in an Anticipatory Learning Classifier System , 2003, ABiALS.

[54]  Stewart W. Wilson.  Function approximation with a classifier system , 2001 .

[55]  Pier Luca Lanzi.  Adding memory to XCS , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[56]  Luca Lanzi Pier,et al.  Extending the Representation of Classifier Conditions Part II: From Messy Coding to S-Expressions , 1999 .

[57]  F. W. Irwin.  Purposive Behavior in Animals and Men , 1932, The Psychological Clinic.

[58]  S. Smith,et al.  A Learning System Based on Genetic Algorithms , 1980 .

[59]  Larry Bull,et al.  A Simple Payoff-Based Learning Classifier System , 2004, PPSN.

[60]  J. P. Seward.  An experimental analysis of latent learning. , 1949, Journal of experimental psychology.

[61]  Terence C. Fogarty,et al.  Co-evolutionary classifier systems for multi-agent simulation , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[62]  Larry Bull,et al.  A Corporate Classifier System , 1998, PPSN.

[63]  Jan Drugowitsch,et al.  Towards convergence of learning classifier systems value iteration , 2006 .

[64]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[65]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[66]  Tim Kovacs,et al.  Advances in Learning Classifier Systems , 2001, Lecture Notes in Computer Science.

[67]  Larry Bull,et al.  Two simple learning classifier systems , 2005 .

[68]  Larry Bull,et al.  Learning Classifier Systems , 2002, Annual Conference on Genetic and Evolutionary Computation.

[69]  Dave Cliff,et al.  Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[70]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[71]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[72]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[73]  C. Sanza.  Evolution d'entités virtuelles coopératives par système de classifieurs , 2001 .

[74]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[75]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[76]  Tim Kovacs.  Strength or accuracy: credit assignment in learning classifier systems , 2003 .

[77]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[78]  Marco Dorigo,et al.  A comparison of Q-learning and classifier systems , 1994 .

[79]  Olivier Sigaud,et al.  Combining latent learning with dynamic programming in the modular anticipatory classifier system , 2005, Eur. J. Oper. Res..

[80]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[81]  Richard S. Sutton,et al.  Planning by Incremental Dynamic Programming , 1991, ML.