Alternatives for Classifier System Credit Assignment

Classifier systems are production rule systems that automatically generate populations of rules cooperating to accomplish desired tasks. The genetic algorithm is the systems' discovery mechanism, and its effectiveness is dependent in part on the accurate estimation of the relative merit of each of the rules (classifiers) in the current population. Merit is estimated conventionally by use of the bucket brigade for credit assignment. This paper addresses the adequacy of the bucket brigade and provides a preliminary exploration of two variants in conjunction with enumerated rules and with discovery. In limited experiments, a variant that combines the bucket brigade, "classifier chunking," and "backwards averaging" has yielded improved performance on simple maze problems. Tentative similarities between this hybrid and Sutton's Adaptive Heuristic Critic (AHC) are suggested.

[1]  Rick L. Riolo,et al.  Bucket Brigade Performance: I. Long Sequences of Classifiers , 1987, ICGA.

[2]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[3]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[4]  Lashon B. Booker,et al.  Intelligent Behavior as an Adaptation to the Task Environment , 1982 .

[5]  Steven Edward Hampson,et al.  A neural model of adaptive behavior , 1983 .

[6]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[8]  A. L. Samuel,et al.  Some studies in machine learning using the game of checkers. II: recent progress , 1967 .

[9]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[10]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[11]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[12]  Ian H. Witten,et al.  An Adaptive Optimal Controller for Discrete-Time Markov Environments , 1977, Inf. Control..

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[15]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[16]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[17]  John H. Holland,et al.  Empirical studies of default hierarchies and sequences of rules in learning classifier systems , 1988 .

[18]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[19]  Gunar E. Liepins,et al.  Machine learning applications to job shop scheduling , 1988, IEA/AIE '88.