Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses

Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability they were applied (to some extent) to Inductive Logic Programming (ILP). Although Estimation of Distribution Algorithms (EDAs) generally perform better than standard GAs, they have not been applied to ILP. This work presents EDA-ILP, an ILP system based on EDA and inverse entailment, and also its extension, the REDA-ILP, which employs the Reduce algorithm in bottom clauses to considerably reduce the search space. Experiments in real-world datasets showed that both systems were successfully compared to Aleph and GA-ILP (another variant of EDA-ILP created replacing the EDA by a standard GA). EDA-ILP was also successfully compared to Progol-QG/GA (and its other variants) in phase transition benchmarks. Additionally, we found that REDA-ILP usually obtains simpler theories than EDA-ILP, more efficiently and with equivalent accuracies. These results show that EDAs provide a good base for stochastic search in ILP.

[1]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[2]  Vítor Santos Costa,et al.  Revising First-Order Logic Theories from Examples Through Stochastic Local Search , 2007, ILP.

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[5]  Stephen Muggleton,et al.  QG/GA: a stochastic search for Progol , 2008, Machine Learning.

[6]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[7]  Céline Rouveirol,et al.  Lazy Propositionalisation for Relational Learning , 2000, ECAI.

[8]  Krzysztof R. Apt,et al.  Logic Programming , 1990, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[9]  Ashwin Srinivasan,et al.  ILP Through Propositionalization and Stochastic k-Term DNF Learning , 2007, ILP.

[10]  Ashwin Srinivasan,et al.  Relating chemical activity to structure: An examination of ILP successes , 1995, New Generation Computing.

[11]  Stephen Muggleton,et al.  TopLog: ILP Using a Logic Program Declarative Bias , 2008, ICLP.

[12]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[13]  Gerson Zaverucha,et al.  Genetic local search for rule learning , 2008, GECCO '08.

[14]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[15]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[16]  Stephen Muggleton,et al.  Searching the Subsumption Lattice by a Genetic Algorithm , 2000, ILP.

[17]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[18]  Jude W. Shavlik,et al.  Using Bayesian Networks to Direct Stochastic Search in Inductive Logic Programming , 2007, ILP.

[19]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[20]  Ivan Bratko,et al.  Refining Complete Hypotheses in ILP , 1999, ILP.

[21]  Ashwin Srinivasan,et al.  Lattice-Search Runtime Distributions May Be Heavy-Tailed , 2002, ILP.

[22]  Jude W. Shavlik,et al.  Gleaner: Creating ensembles of first-order clauses to improve recall-precision curves , 2006, Machine Learning.

[23]  Raymond J. Mooney,et al.  Discriminative structure and parameter learning for Markov logic networks , 2008, ICML '08.

[24]  Aomar Osmani,et al.  On the connection between the phase transition of the covering test and the learning success rate in ILP , 2008, Machine Learning.

[25]  Ashwin Srinivasan,et al.  Carcinogenesis Predictions Using ILP , 1997, ILP.

[26]  Michèle Sebag,et al.  Relational Learning as Search in a Critical Region , 2003, J. Mach. Learn. Res..

[27]  Gerson Zaverucha,et al.  Inductive Logic Programming through Estimation of Distribution Algorithm , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[28]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .