Relational concept learning by cooperative evolution

Concept learning is a computationally demanding task that involves searching large hypothesis spaces containing candidate descriptions. Stochastic search combined with parallel processing provide a promising approach to successfully deal with such computationally intensive tasks.Learning systems based on distributed genetic algorithms (GA) were able to find concept descriptions as accurate as the ones found by state-of-the-art learning systems based on alternative approaches. However, genetic algorithms' exploitation has the drawback of being computationally demanding.We show how a suitable architectural choice, named cooperative evolution, allows to solve complex applications in an acceptable user waiting time and with a reasonable computational cost by using GA-based learning systems because of the effective exploitation of distributed computation. A variety of experimental settings is analyzed and an explanation for the empirical observations is proposed.

[1]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[2]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[3]  Maurice Bruynooghe,et al.  Declarative bias for specific-to-general ILP systems , 1995, Machine Learning.

[4]  John J. Grefenstette,et al.  A Coevolutionary Approach to Learning Sequential Decision Rules , 1995, ICGA.

[5]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[6]  Andrew McCallum,et al.  Using Genetic Algorithms to Learn Disjunctive Rules from Examples , 1990, ML.

[7]  Mitchell A. Potter,et al.  The design and analysis of a computational model of cooperative coevolution , 1997 .

[8]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[9]  Jukka Hekanaho.  Background Knowledge in GA-based Concept Learning , 1996, ICML.

[10]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[11]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[12]  János Csirik,et al.  Learning Phonetic Rules in a Speech Recognition System , 1997, ILP.

[13]  Jonathan L. Shapiro,et al.  Does Data-Model Co-evolution Improve Generalization Performance of Evolving Learners? , 1998, PPSN.

[14]  David W. Aha,et al.  Towards a Better Understanding of Memory-based Reasoning Systems , 1994, ICML.

[15]  Filippo Neri,et al.  Search-Intensive Concept Induction , 1995, Evolutionary Computation.

[16]  Michèle Sebag,et al.  Tractable Induction and Classification in First Order Logic Via Stochastic Matching , 1997, IJCAI.

[17]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[18]  Ryszard S. Michalski,et al.  A theory and methodology of inductive learning , 1993 .

[19]  F. Neri,et al.  Comparing local search with respect to genetic evolution to detect intrusions in computer networks , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[20]  David E. Goldberg,et al.  Implicit Niching in a Learning Classifier System: Nature's Way , 1994, Evolutionary Computation.

[21]  Filippo Neri,et al.  An Analysis of the Universal Suffrage Selection Operator , 1996, Evolutionary Computation.

[22]  Richard A. Lewis,et al.  Drug design by machine learning: the use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[23]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[24]  Robert E. Schapire,et al.  A Brief Introduction to Boosting , 1999, IJCAI.

[25]  Chrisila C. Pettey,et al.  A Theoretical Investigation of a Parallel Genetic Algorithm , 1989, ICGA.

[26]  Gilles Venturini,et al.  SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes based Concepts , 1993, ECML.

[27]  Sholom M. Weiss,et al.  Small Sample Decision tree Pruning , 1994, ICML.

[28]  G DietterichThomas.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees , 2000 .

[29]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[30]  Salvatore J. Stolfo,et al.  Mining Audit Data to Build Intrusion Detection Models , 1998, KDD.

[31]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[32]  Jack Dongarra,et al.  PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing , 1995 .

[33]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[34]  Cezary Z. Janikow,et al.  A knowledge-intensive genetic algorithm for supervised learning , 1993, Machine Learning.

[35]  Ashwin Srinivasan,et al.  Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction , 1996, Artif. Intell..

[36]  Filippo Neri,et al.  Exploring the Power of Genetic Search in Learning Symbolic Classifiers , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[38]  Luc De Raedt,et al.  A Theory of Clausal Discovery , 1993, IJCAI.

[39]  M. Pazzani,et al.  The Utility of Knowledge in Inductive Learning , 1992, Machine Learning.

[40]  R. Mike Cameron-Jones,et al.  Oversearching and Layered Search in Empirical Learning , 1995, IJCAI.

[41]  Robert C. Holte,et al.  Concept Learning and the Problem of Small Disjuncts , 1989, IJCAI.

[42]  Thomas G. Dietterich,et al.  A Comparative Review of Selected Methods for Learning from Examples , 1983 .

[43]  Stephen F. Smith,et al.  Flexible Learning of Problem Solving Heuristics Through Adaptive Search , 1983, IJCAI.

[44]  Kenneth A. De Jong,et al.  Using genetic algorithms for concept learning , 1993, Machine Learning.

[45]  Filippo Neri,et al.  Integrating Multiple Learning Strategies in First Order Logics , 1997, Machine Learning.