Can predicate invention compensate for incomplete background knowledge?

In machine learning we are often faced with the problem of incomplete data, which can lead to lower predictive accuracies in both feature-based and relational machine learning. It is therefore important to develop techniques to compensate for incomplete data. In inductive logic programming (ILP) incomplete data can be in the form of missing values or missing predicates. In this paper, we investigate whether an ILP learner can compensate for missing background predicates through predicate invention. We conduct experiments on two datasets in which we progressively remove predicates from the background knowledge whilst measuring the predictive accuracy of three ILP learners with differing levels of predicate invention. The experimental results show that as the number of background predicates decreases, an ILP learner which performs predicate invention has higher predictive accuracies than the learners which do not perform predicate invention, suggesting that predicate invention can compensate for incomplete background knowledge.

[1]  Johannes Fürnkranz,et al.  Dimensionality Reduction in ILP: A Call to Arms , 1997 .

[2]  Stephen Muggleton,et al.  Variation of Background Knowledge in an Industrial Application of ILP , 2010, ILP.

[3]  Stephen Muggleton,et al.  Logical Minimisation of Meta-Rules Within Meta-Interpretive Learning , 2014, ILP.

[4]  Stephen Muggleton,et al.  Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited , 2013, Machine Learning.

[5]  Stephen Muggleton,et al.  Bias reformulation for one-shot function induction , 2014, ECAI.

[6]  Nada Lavrač.  Handling Imperfect Data in Inductive Logic Programming , 1996 .

[7]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[8]  Vipin Kumar.  UNDERSTANDING COMPLEX DATASETS: DATA MINING WITH MATRIX DECOMPOSITIONS , 2006 .

[9]  Stephen Muggleton,et al.  Inductive Logic Programming , 2011, Lecture Notes in Computer Science.

[10]  Robin Parker,et al.  Missing Data Problems in Machine Learning , 2010 .

[11]  Ashwin Srinivasan,et al.  Mutagenesis: ILP experiments in a non-determinate biological domain , 1994 .

[12]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[13]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[14]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[15]  G. Plotkin.  Automatic Methods of Inductive Inference , 1972 .

[16]  Stephen Muggleton,et al.  Predicate invention and utilization , 1994, J. Exp. Theor. Artif. Intell..

[17]  Stephen Muggleton,et al.  Meta-interpretive learning: application to grammatical inference , 2013, Machine Learning.

[18]  Michael I. Jordan,et al.  Learning from Incomplete Data , 1994 .

[19]  Luc De Raedt.  Declarative Modeling for Machine Learning and Data Mining , 2012, Discovery Science.

[20]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[21]  Sašo Džeroski,et al.  Handling imperfect data in inductive logic programming , 1993 .

[22]  Irene Stahl,et al.  Predicate Invention in ILP - an Overview , 1993, ECML.

[23]  Ning Zhong,et al.  Rough Problem Settings for Inductive Logic Programming , 1999, RSFDGrC.