Scaling Up Inductive Logic Programming: An Evolutionary Wrapper Approach

Inductive logic programming (ILP) algorithms are classification algorithms that construct classifiers represented as logic programs. ILP algorithms have a number of attractive features, notably the ability to make use of declarative background (user-supplied) knowledge. However, ILP algorithms deal poorly with large data sets (>104 examples) and their widespread use of the greedy set-covering algorithm renders them susceptible to local maxima in the space of logic programs.This paper presents a novel approach to address these problems based on combining the local search properties of an inductive logic programming algorithm with the global search properties of an evolutionary algorithm. The proposed algorithm may be viewed as an evolutionary wrapper around a population of ILP algorithms.The evolutionary wrapper approach is evaluated on two domains. The chess-endgame (KRK) problem is an artificial domain that is a widely used benchmark in inductive logic programming, and Part-of-Speech Tagging is a real-world problem from the field of Natural Language Processing. In the latter domain, data originates from excerpts of the Wall Street Journal. Results indicate that significant improvements in predictive accuracy can be achieved over a conventional ILP approach when data is plentiful and noisy.

[1]  William E. Hart,et al.  Optimization with genetic algorithm hybrids that use local searches , 1996 .

[2]  Peter A. Whigham,et al.  Time series modeling using genetic programming: an application to rainfall-runoff models , 1999 .

[3]  James Cussens.  Part-of-Speech Tagging Using Progol , 1997, ILP.

[4]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[5]  Frédéric Gruau,et al.  On using syntactic constraints with genetic programming , 1996 .

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  Filippo Neri.  Cooperative Concept Learning By Means Of A Distributed GA , 2002, GECCO.

[8]  Roger L. Wainwright,et al.  Type inheritance in strongly typed genetic programming , 1996 .

[9]  Stephen Muggleton.  Inductive Logic Programming: Derivations, Successes and Shortcomings , 1993, ECML.

[10]  Peter J. Angeline,et al.  Evolving Recursive Functions for the Even-Parity Problem Using Genetic Programming , 1996 .

[11]  Kwong-Sak Leung,et al.  Medical Data Mining Using Evolutionary Computation , 2022 .

[12]  Gilles Venturini,et al.  Learning First Order Logic Rules with a Genetic Algorithm , 1995, KDD.

[13]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[14]  P MarcusMitchell,et al.  Building a large annotated corpus of English , 1993 .

[15]  David E. Goldberg,et al.  Genetic and evolutionary algorithms come of age , 1994, CACM.

[16]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[17]  Kwong-Sak Leung,et al.  Combining genetic programming and inductive logic programming using logic grammars , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[18]  John J. Grefenstette,et al.  The Evolution of Strategies for Multiagent Environments , 1992, Adapt. Behav..

[19]  Stephen Muggleton,et al.  An Experimental Comparison of Human and Machine Learning Formalisms , 1989, ML.

[20]  John J. Grefenstette.  Learning Decision Strategies with Genetic Algorithms , 1992, AII.

[21]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[22]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[23]  C. Janikow.  A Knowledge-Intensive Genetic Algorithm for Supervised Learning , 2004, Machine Learning.

[24]  John J. Greffenstette,et al.  A System for Learning Control Strategies with Genetic Algorithms , 1989 .

[25]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[26]  Kenneth A. De Jong,et al.  Using Genetic Algorithms to Solve NP-Complete Problems , 1989, ICGA.

[27]  Kwong-Sak Leung,et al.  An induction system that learns programs in different programming languages using genetic programming and logic grammars , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[28]  Nostrand Reinhold,et al.  the utility of using the genetic algorithm approach on the problem of Davis, L. (1991), Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York. , 1991 .

[29]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[30]  Kwong-Sak Leung,et al.  Applying logic grammars to induce sub-functions in genetic programming , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[31]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[32]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[33]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[34]  Lorenza Saitta,et al.  Learning Disjunctive Concepts by Means of Genetic Algorithms , 1994, ICML.

[35]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[36]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[37]  Matevz Kovacic.  Stochastic Inductive Logic Programming , 1994 .

[38]  Peter A. Whigham,et al.  Search bias, language bias and genetic programming , 1996 .

[39]  Kenneth A. De Jong,et al.  Using genetic algorithms for concept learning , 1993, Machine Learning.

[40]  John J. Grefenstette,et al.  A System for Learning Control Strategies with Genetic Algorithms , 1989, ICGA.

[41]  John J. Grefenstette,et al.  Improving tactical plans with genetic algorithms , 1990, [1990] Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence.

[42]  Thomas G. Dietterich.  Machine-Learning Research Four Current Directions , 1997 .

[43]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[44]  Igor Kononenko,et al.  SFOIL: Stochastic Approach to Inductive Logic Programming , 1993 .