Credit Assignment in Rule Discovery Systems Based on Genetic Algorithms

In rule discovery systems, learning often proceeds by first assessing the quality of the system's current rules and then modifying rules based on that assessment. This paper addresses the credit assignment problem that arises when long sequences of rules fire between successive external rewards. The focus is on the kinds of rule assessment schemes which have been proposed for rule discovery systems that use genetic algorithms as the primary rule modification strategy. Two distinct approaches to rule learning with genetic algorithms have been previously reported, each approach offering a useful solution to a different level of the credit assignment problem. We describe a system, called RUDI, that exploits both approaches. We present analytic and experimental results that support the hypothesis that multiple levels of credit assignment can improve the performance of rule learning systems based on genetic algorithms.

[1]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[2]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[3]  Donald A. Waterman,et al.  Generalization Learning Techniques for Automating the Learning of Heuristics , 1970, Artif. Intell..

[4]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[5]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[6]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[7]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[8]  Tom M. Mitchell,et al.  Learning from Solution Paths: An Approach to the Credit Assignment Problem , 1982, AI Mag..

[9]  Lashon B. Booker,et al.  Intelligent Behavior as an Adaptation to the Task Environment , 1982 .

[10]  Stephen F. Smith,et al.  Flexible Learning of Problem Solving Heuristics Through Adaptive Search , 1983, IJCAI.

[11]  Pat Langley,et al.  Learning Effective Search Heuristics , 1983, IJCAI.

[12]  Stewart W. Wilson.  Knowledge Growth in an Artificial Animal , 1985, ICGA.

[13]  J. David Schaffer,et al.  Multi-Objective Learning via Genetic Algorithms , 1985, IJCAI.

[14]  John H. Holland,et al.  Properties of the bucket brigade algorithm , 1985 .

[15]  Thomas H. Westerdale,et al.  The Bucket Brigade Is Not Genetic , 1985, International Conference on Genetic Algorithms.

[16]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[17]  Tom Michael Mitchell,et al.  Explanation-based generalization: A unifying view , 1986 .

[18]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  Kenneth A. De Jong,et al.  On Using Genetic Algorithms to Search Program Spaces , 1987, ICGA.

[20]  Rick L. Riolo,et al.  Bucket Brigade Performance: I. Long Sequences of Classifiers , 1987, ICGA.

[21]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[22]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[23]  John J. Grefenstette,et al.  Csm: a genetic classifier system with memory for learning by analogy , 1987 .

[24]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[25]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[26]  Tom M. Mitchell,et al.  Learning by experimentation: acquiring and refining problem-solving heuristics , 1993 .

[27]  Stewart W. Wilson.  Classifier systems and the animat problem , 2004, Machine Learning.

[28]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.