Competition-Based Induction of Decision Models from Examples

Symbolic induction is a promising approach to constructing decision models by extracting regularities from a data set of examples. The predominant type of model is a classification rule (or set of rules) that maps a set of relevant environmental features into specific categories or values. Classifying loan risk based on borrower profiles, consumer choice from purchase data, or supply levels based on operating conditions are all examples of this type of model-building task. Although current inductive approaches, such as ID3 and CN2, perform well on certain problems, their potential is limited by the incremental nature of their search. Genetic algorithms (GA) have shown great promise on complex search domains, and hence suggest a means for overcoming these limitations. However, effective use of genetic search in this context requires a framework that promotes the fundamental model-building objectives of predictive accuracy and model simplicity. In this article we describe COGIN, a GA-based inductive system that exploits the conventions of induction from examples to provide this framework. The novelty of COGIN lies in its use of training set coverage to simultaneously promote competition in various classification niches within the model and constrain overall model complexity. Experimental comparisons with NewID and CN2 provide evidence of the effectiveness of the COGIN framework and the viability of the GA approach.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  Lashon B. Booker,et al.  Intelligent Behavior as an Adaptation to the Task Environment , 1982 .

[3]  Stephen F. Smith,et al.  Flexible Learning of Problem Solving Heuristics Through Adaptive Search , 1983, IJCAI.

[4]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[5]  J. D. Schaffer,et al.  Some experiments in machine learning using vector evaluated genetic algorithms (artificial intelligence, optimization, adaptation, pattern recognition) , 1984 .

[6]  David E. Goldberg,et al.  Dynamic System Control Using Rule Learning and Genetic Algorithms , 1985, IJCAI.

[7]  James E. Baker,et al.  Adaptive Selection Methods for Genetic Algorithms , 1985, International Conference on Genetic Algorithms.

[8]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[9]  Stephen F. Smith,et al.  A Genetic System for Learning Models of Consumer Choice , 1987, ICGA.

[10]  David Perry Greene.  Automated Knowledge Acquisition: Overcoming the Expert System Bottleneck , 1987, ICIS.

[11]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[12]  Ian H. Witten,et al.  A framework for knowledge acquisition through techniques of concept learning , 1989, IEEE Trans. Syst. Man Cybern..

[13]  David E. Goldberg,et al.  A Critical Review of Classifier Systems , 1989, ICGA.

[14]  Lashon B. Booker,et al.  Triggered Rule Discovery in Classifier Systems , 1989, ICGA.

[15]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[16]  Ryszard S. Michalski,et al.  Machine learning: an artificial intelligence approach volume III , 1990 .

[17]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[18]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[19]  John R. Koza,et al.  Evolving a Computer Program to Generate Random Numbers Using the Genetic Programming Paradigm , 1991, ICGA.

[20]  Stephen F. Smith,et al.  COGIN: Symbolic Induction with Genetic Algorithms , 1992, AAAI.

[21]  John J. Grefenstette,et al.  Learning sequential decision rules using simulation models and competition , 2004, Machine Learning.

[22]  Peter Clark,et al.  The CN2 induction algorithm , 2004, Machine Learning.

[23]  G. Robertson,et al.  A Tale of Two Classifier Systems , 2005, Machine Learning.

[24]  Stewart W. Wilson.  Classifier Systems and the Animat Problem , 1987, Machine Learning.