A Roadmap to the Last Decade of Learning Classifier System Research

In 1989 Wilson and Goldberg presented a critical review of the first ten years of learning classifier system research With this paper we review the subsequent ten years of learning classifier systems research discussing the main achievements and the major research directions pursued in those years.

[1]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[2]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[3]  Philippe Collard,et al.  Relational Schemata: A Way to Improve the Expressiveness of Classifiers , 1995, ICGA.

[4]  Lashon B. Booker,et al.  Do We Really Need to Estimate Rule Utilities in Classifier Systems? , 1999, Learning Classifier Systems.

[5]  Stewart W. Wilson.  Get Real! XCS with Continuous-Valued Inputs , 1999, Learning Classifier Systems.

[6]  Pattie Maes,et al.  Spatial exploration, map learning, and self-positioning with MonaLysa , 1996 .

[7]  Larry Bull,et al.  A Corporate XCS , 1999, Learning Classifier Systems.

[8]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[9]  David E. Goldberg,et al.  Variable Default Hierarchy Separation in a Classifier System , 1990, FOGA.

[10]  Stewart W. Wilson.  Generalization in the XCS Classifier System , 1998 .

[11]  Adrian R. Hartley,et al.  Accuracy-based fitness allows similar performance to humans in static and dynamic classification environments , 1999, GECCO.

[12]  Lashon B. Booker,et al.  Representing Attribute-Based Concepts in a Classifier System , 1990, FOGA.

[13]  Nicolaas J. Vriend,et al.  An Illustration of the Essential Difference between Individual and Social Learning, and its Consequences for Computational Analyses , 1998 .

[14]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[15]  Marco Dorigo,et al.  Adaptive Learning of a Robot Arm , 1994, Evolutionary Computing, AISB Workshop.

[16]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[17]  Stewart W. Wilson,et al.  Toward Optimal Classifier System Performance in Non-Markov Environments , 2000, Evolutionary Computation.

[18]  Rick L. Riolo,et al.  Modeling Simple Human Category Learning with a Classifier System , 1991, International Conference on Genetic Algorithms.

[19]  Francine Federman,et al.  Information Theory and NEXTPITCH: A Learning Classifier System , 1997, ICGA.

[20]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[21]  S. D. Sheppard,et al.  TWO-DIMENSIONAL STRUCTURAL COMPONENT SHAPE IMPROVEMENT VIA CLASSIFIER SYSTEM , 1992 .

[22]  David E. Goldberg,et al.  Reinforcement learning with classifier systems: Adaptive default hierarchy formation , 1992, Appl. Artif. Intell..

[23]  Dave Cliff,et al.  Adding Temporary Memory to ZCS , 1994, Adapt. Behav..

[24]  Rick L. Riolo,et al.  The Emergence of Coupled Sequences of Classifiers , 1989, ICGA.

[25]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[26]  Larry Bull,et al.  A Corporate Classifier System , 1998, PPSN.

[27]  Pattie Maes,et al.  Explore/Exploit Strategies in Autonomy , 1996 .

[28]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[29]  John R. Koza,et al.  Hidden Order: How Adaptation Builds Complexity. , 1995, Artificial Life.

[30]  R. Palmer,et al.  Time series properties of an artificial stock market , 1999 .

[31]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[32]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[33]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[34]  Alwyn Barry,et al.  Aliasing in XCS and the Consecutive State Problem: 2 - Solutions , 1999, GECCO.

[35]  Jonathan Schaeffer,et al.  VCS: Variable Classifier Systems , 1989, ICGA.

[36]  Robert E. Smith,et al.  The Fighter Aircraft LCS: A Case of Different LCS Goals and Techniques , 1999, Learning Classifier Systems.

[37]  Gary Roberts,et al.  A Rational Reconstruction of Wilson's Animat and Holland's CS-1 , 1989, International Conference on Genetic Algorithms.

[38]  Robert E. Smith,et al.  Memory Exploitation in Learning Classifier Systems , 1994, Evolutionary Computation.

[39]  L. Booker.  Classifier Systems that Learn Internal World Models , 2005, Machine Learning.

[40]  Marco Dorigo,et al.  Alecsys: A Parallel Laboratory for Learning Classifier Systems , 1991, ICGA.

[41]  Diego Calvanese,et al.  Unifying Class-Based Representation Formalisms , 2011, J. Artif. Intell. Res..

[42]  T. H. Westerdale.  An approach to credit assignment in classifier systems , 1998, Complex..

[43]  John H. Holmes,et al.  Learning Classifier Systems Applied to Knowledge Discovery in Clinical Research Databases , 1999, Learning Classifier Systems.

[44]  Larry Bull,et al.  Design of a Traffic Junction Controller Using Classifier Systems and Fuzzy Logic , 1999, Fuzzy Days.

[45]  Alwyn Barry,et al.  Aliasing in XCS and the Consecutive State Problem: 1 - Effects , 1999, GECCO.

[46]  Marco Dorigo,et al.  Using transputers to increase speed and flexibility of genetics-based machine learning systems , 1992, Microprocess. Microprogramming.

[47]  Rick L. Riolo,et al.  Lookahead planning and latent learning in a classifier system , 1991 .

[48]  John H. Holland,et al.  Hidden Order: How Adaptation Builds Complexity , 1995 .

[49]  Johann Mitlöhner.  Classifier systems and economic modeling , 1996 .

[50]  Marco Colombetti,et al.  Behavior analysis and training-a methodology for behavior engineering , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[51]  Dipankar Dasgupta,et al.  Metacognition in Software Agents Using Classifier Systems , 1998, AAAI/IAAI.

[52]  Marco Colombetti,et al.  Robot Shaping: Developing Autonomous Agents Through Learning , 1994, Artif. Intell..

[53]  R. Palmer,et al.  Asset Pricing Under Endogenous Expectations in an Artificial Stock Market , 1996 .

[54]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[55]  Jean-Arcady Meyer,et al.  From Animals to Animats: Proceedings of The First International Conference on Simulation of Adaptive Behavior (Complex Adaptive Systems) , 1990 .

[56]  Sean Saxon,et al.  XCS and the Monk's Problems , 1999, Learning Classifier Systems.

[57]  Stewart W. Wilson.  Classifier System Mapping of Real Vectors , 1992 .

[58]  Stewart W. Wilson.  Knowledge Growth in an Artificial Animal , 1985, ICGA.

[59]  Robert A. Richards,et al.  A Learning Classifier System for Three-Dimensional Shape Optimization , 1996, PPSN.

[60]  Pier Luca Lanzi.  Adding memory to XCS , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[61]  Elisabet Golobardes,et al.  Automatic diagnosis with genetic algorithms and case-based reasoning , 1999, Artif. Intell. Eng..

[62]  Andrea Bonarini,et al.  An Introduction to Learning Fuzzy Classifier Systems , 1999, Learning Classifier Systems.

[63]  Luca Lanzi Pier,et al.  Extending the Representation of Classifier Conditions Part II: From Messy Coding to S-Expressions , 1999 .

[64]  T. H. Westerdale.  An approach to credit assignment in classifier systems , 1998 .

[65]  Lashon B. Booker,et al.  Instinct as an inductive bias for learning behavioral sequences , 1991 .

[66]  P. Lanzi.  Extending the representation of classifier conditions part I: from binary to messy coding , 1999 .

[67]  Michael Davis,et al.  A computational model of affect theory: Simulations of reducer/augmenter and learned helplessness phenomena. , 2000 .

[68]  John H. Holmes.  Evaluating learning classifier system performance in two-choice decision tasks: an LCS metric toolkit , 1999 .

[69]  Sandip Sen,et al.  Newboole: A Fast GBML System , 1990, ML.

[70]  John H. Holmes.  Discovering Risk of Disease with a Learning Classifier System , 1997, ICGA.

[71]  Josep Maria Garrell i Guiu,et al.  Auotmatic classification of mammary biopsy images with machine learning techniques , 1998 .

[72]  Rick L. Riolo,et al.  Bucket Brigade Performance: II. Default Hierarchies , 1987, ICGA.

[73]  John R. Koza,et al.  Genetic programming (videotape): the movie , 1992 .

[74]  Rick L. Riolo,et al.  The Emergence of Default Hierarchies in Learning Classifier Systems , 1989, ICGA.

[75]  Wolfgang Stolzmann,et al.  An Introduction to Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[76]  Dale Schuurmans,et al.  Representational Difficulties with Classifier Systems , 1989, ICGA.

[77]  Larry Bull,et al.  On corporate classifier systems: Increasing the benefits of rule linkage , 1999 .

[78]  David E. Goldberg,et al.  Implicit Niching in a Learning Classifier System: Nature's Way , 1994, Evolutionary Computation.

[79]  Marco Dorigo,et al.  Genetic and Non-Genetic Operators in ALECSYS , 1993, Evolutionary Computation.

[80]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[81]  Teresa Satterfield.  Bilingual Selection of Syntactic Knowledge: Extending the Principles and Parameters Approach , 1999 .

[82]  T. Kovacs.  Deletion schemes for classifier systems , 1999 .

[83]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[84]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[85]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[86]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[87]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[88]  Frederick Hayes-Roth.  Patterns of induction and associated knowledge acquisition algorithms , 1976 .

[89]  Marco Colombetti,et al.  Robot shaping: The Hamster Experiment , 1996 .

[90]  Dijia Huang.  The Context-Array Bucket-Brigade Algorithm: An Enhanced Approach to Credit-Apportionment in Classifier Systems , 1989, ICGA.

[91]  T. Kovacs.  XCS Classifier System Reliably Evolves Accurate, Complete, and Minimal Representations for Boolean Functions , 1998 .

[92]  Sandip Sen,et al.  A tale of two representations , 1994, IEA/AIE '94.

[93]  Ellen R. McGrattan,et al.  Money as a medium of exchange in an economy with artificially intelligent agents , 1990 .

[94]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[95]  Marco Dorigo,et al.  Alecsys and the AutonoMouse: Learning to control a real robot by distributed classifier systems , 2004, Machine Learning.

[96]  J. Davenport.  Editor , 1960 .

[97]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[98]  Sandip Sen,et al.  Modeling Human Categorization By a Simple Classifier System , 1996 .

[99]  John H. Holland,et al.  Empirical studies of default hierarchies and sequences of rules in learning classifier systems , 1988 .

[100]  Marco Dorigo,et al.  A comparison of Q-learning and classifier systems , 1994 .

[101]  Marco Colombetti,et al.  Training Agents to Perform Sequential Behavior , 1994, Adapt. Behav..

[102]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[103]  J. van Leeuwen,et al.  Learning Classifier Systems , 2000, Lecture Notes in Computer Science.

[104]  Stewart W. Wilson.  State of XCS Classifier System Research , 1999, Learning Classifier Systems.

[105]  Gary Roberts.  Dynamic Planning for Classifier Systems , 1993, ICGA.

[106]  Robert E. Smith,et al.  Classifier systems in combat: two-sided learning of maneuvers for advanced fighter aircraft , 2000 .

[107]  Larry Bull,et al.  A zeroth level corporate classifier system , 1999 .

[108]  J. Holmes.  Evolution-assisted discovery of sentinel features in epidemiologic surveillance , 1996 .

[109]  Marco Colombetti,et al.  Robot Shaping: An Experiment in Behavior Engineering , 1997 .

[110]  A. H. Gilbert,et al.  Adaptive Learning of Process Control and Profit Optimization Using a Classifier System , 1995, Evolutionary Computation.

[111]  Lashon B. Booker,et al.  Triggered Rule Discovery in Classifier Systems , 1989, ICGA.

[112]  Marco Colombetti,et al.  Evolutionary Computation in Behavior Engineering , 1999 .

[113]  Robert A. Richards,et al.  Zeroth-order shape optimization utilizing a learning classifier system , 1996 .

[114]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[115]  K. Holyoak,et al.  A Theory of Conditioning: Inductive Learning within Rule-Based Default Hierarchies. , 1989 .

[116]  Robert A. Richards,et al.  Three-dimensional shape optimization utilizing a learning classifier system , 1996 .

[117]  Gunar E. Liepins,et al.  Alternatives for Classifier System Credit Assignment , 1989, IJCAI.

[118]  Jean-Arcady Meyer,et al.  A hierarchical classifier system implementing a motivationally autonomous animat , 1994 .

[119]  Stewart W. Wilson.  Bid Competition and Specificity Reconsidered , 1988, Complex Syst..

[120]  Larry Bull,et al.  A Genetic Programming-based Classifier System , 1999, GECCO.

[121]  J. Holland,et al.  Artificial Adaptive Agents in Economic Theory , 1991 .

[122]  Francine Federman,et al.  Representation of Music in a Learning Classifier System , 1997, ISMIS.

[123]  Alexandre Parodi,et al.  An Efficient Classifier System and Its Experimental Comparison with Two Representative Learning Methods on Three Medical Domains , 1991, ICGA.

[124]  David E. Moriarty,et al.  Eecient Reinforcement Learning through Symbiotic Evolution Editor: Leslie Pack Kaelbling , 1996 .

[125]  Martin V. Butz,et al.  Latent Learning and Action Planning in Robots with Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[126]  Tim Kovacs,et al.  Strength or Accuracy? Fitness Calculation in Learning Classifier Systems , 1999, Learning Classifier Systems.

[127]  Stewart W. Wilson.  ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[128]  Larry Bull,et al.  On using ZCS in a Simulated Continuous Double-Auction Market , 1999, GECCO.

[129]  Jean-Arcady Meyer,et al.  Hierarchical Map Building and Self-Positioning with MonaLysa , 1996, Adapt. Behav..

[130]  Gunar E. Liepins,et al.  Credit assignment and discovery in classifier systems , 1991, Int. J. Intell. Syst..

[131]  Teresa Satterfield,et al.  Bilingual Selection of Syntactic Knowledge , 1999 .

[132]  D. Goldberg,et al.  Natural niching for evolving cooperative classifiers , 1996 .

[133]  Tim Kovacs,et al.  A Learning Classifier Systems Bibliography , 1999, Learning Classifier Systems.

[134]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[135]  Marco Dorigo,et al.  Genetics-based machine learning and behavior-based robotics: a new synthesis , 1993, IEEE Trans. Syst. Man Cybern..