Probabilistic Inductive Logic Programming

Probabilistic inductive logic programming aka. statistical relational learning addresses one of the central questions of artificial intelligence: the integration of probabilistic reasoning with machine learning and first order and relational logic representations. A rich variety of different formalisms and learning techniques have been developed. A unifying characterization of the underlying learning settings, however, is missing so far. In this chapter, we start from inductive logic programming and sketch how the inductive logic programming formalisms, settings and techniques can be extended to the statistical case. More precisely, we outline three classical settings for inductive logic programming, namely learning from entailment, learning from interpretations, and learning from proofs or traces, and show how they can be adapted to cover state-of-the-art statistical relational learning approaches.

[1]  Luc De Raedt,et al.  Towards Combining Inductive Logic Programming with Bayesian Networks , 2001, ILP.

[2]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[3]  Victor W. Marek.  Book review: The Art of Prolog Advanced Programming Techniques by L. Sterling and E. Shapiro (The MIT Press) , 1988, SGAR.

[4]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[5]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[6]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[7]  L. De Raedt,et al.  Logical Hidden Markov Models , 2011, J. Artif. Intell. Res..

[8]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[9]  Jennifer Neville,et al.  Dependency networks for relational data , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[10]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[11]  Luc De Raedt,et al.  First-Order jk-Clausal Theories are PAC-Learnable , 1994, Artif. Intell..

[12]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[13]  Luc De Raedt,et al.  nFOIL: Integrating Naïve Bayes and FOIL , 2005, AAAI.

[14]  Peter Haddawy,et al.  Answering Queries from Context-Sensitive Probabilistic Knowledge Bases , 1997, Theor. Comput. Sci..

[15]  Luc De Raedt,et al.  Logical Settings for Concept-Learning , 1997, Artif. Intell..

[16]  Taisuke Sato,et al.  A Statistical Learning Method for Logic Programs with Distribution Semantics , 1995, ICLP.

[17]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[18]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[19]  Luc De Raedt,et al.  Clausal Discovery , 1997, Machine Learning.

[20]  Avi Pfeffer,et al.  Learning Probabilities for Noisy First-Order Rules , 1997, IJCAI.

[21]  Andreas Stolcke,et al.  Inducing Probabilistic Grammars by Bayesian Model Merging , 1994, ICGI.

[22]  Pedro M. Domingos,et al.  Relational Markov models and their application to adaptive web navigation , 2002, KDD.

[23]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[24]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[25]  Stephen Muggleton,et al.  Learning Stochastic Logic Programs , 2000, Electron. Trans. Artif. Intell..

[26]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[27]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[28]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[29]  Peter A. Flach.  Simply logical - intelligent reasoning by example , 1994, Wiley professional computing.

[30]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[31]  Ben Taskar,et al.  Learning Probabilistic Models of Link Structure , 2003, J. Mach. Learn. Res..

[32]  R. Mike Cameron-Jones,et al.  Induction of logic programs: FOIL and related systems , 1995, New Generation Computing.

[33]  Peter A. Flach,et al.  Naive Bayesian Classification of Structured Data , 2004, Machine Learning.

[34]  Luc De Raedt,et al.  Towards Discovering Structural Signatures of Protein Folds Based on Logical Hidden Markov Models , 2003, Pacific Symposium on Biocomputing.

[35]  Maurice Bruynooghe,et al.  Logic programs with annotated disjunctions , 2004, NMR.

[36]  Yoshitaka Kameya,et al.  Parameter Learning of Logic Programs for Symbolic-Statistical Modeling , 2001, J. Artif. Intell. Res..

[37]  Steven P. Abney.  Stochastic Attribute-Value Grammars , 1996, CL.

[38]  James Cussens.  Loglinear models for first-order probabilistic reasoning , 1999, UAI.

[39]  Daphne Koller,et al.  Probabilistic reasoning for complex systems , 1999 .

[40]  Peter Haddawy,et al.  Generating Bayesian Networks from Probablity Logic Knowledge Bases , 1994, UAI.

[41]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[42]  Thomas D. Nielsen,et al.  Structural Learning in Object Oriented Domains , 2001, FLAIRS.

[43]  De Raedt,et al.  Advances in Inductive Logic Programming , 1996 .

[44]  Tapani Raiko,et al.  "Say EM" for Selecting Probabilistic Models for Logical Sequences , 2005, UAI.

[45]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[46]  Andreas Stolcke,et al.  Hidden Markov Model} Induction by Bayesian Model Merging , 1992, NIPS.

[47]  Leon Sterling,et al.  The Art of Prolog - Advanced Programming Techniques , 1986 .

[48]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[49]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[50]  Stephen Muggleton,et al.  Learning Structure and Parameters of Stochastic Logic Programs , 2002, ILP.

[51]  James Cussens,et al.  Parameter Estimation in Stochastic Logic Programs , 2001, Machine Learning.

[52]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[53]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .

[54]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[55]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[56]  Neng-Fa Zhou,et al.  Yet More Efficient EM Learning for Parameterized Logic Programs by Inter-Goal Sharing , 2004, ECAI.

[57]  Francesco Bergadano,et al.  Inductive Logic Programming: From Machine Learning to Software Engineering , 1995 .

[58]  Nicolas Helft,et al.  Induction as Nonmonotonic Inference , 1989, KR.

[59]  Ben Taskar,et al.  Markov Logic: A Unifying Framework for Statistical Relational Learning , 2007 .

[60]  Stig K. Andersen,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[61]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[62]  James Cussens.  Integrating by Separating : Combining Probability and Logic with ICL , PRISM and SLPs , 2005 .

[63]  Luc De Raedt,et al.  Adaptive Bayesian Logic Programs , 2001, ILP.

[64]  Luc De Raedt,et al.  Probabilistic logic learning , 2003, SKDD.

[65]  Gordon Plotkin,et al.  A Note on Inductive Generalization , 2008 .

[66]  Fabrizio Riguzzi,et al.  Learning Logic Programs with Annotated Disjunctions , 2004, ILP.

[67]  Luc De Raedt,et al.  Towards Learning Stochastic Logic Programs from Proof-Banks , 2005, AAAI.

[68]  Luc De Raedt,et al.  Bayesian Logic Programming: Theory and Tool , 2007 .

[69]  Ashwin Srinivasan,et al.  Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction , 1996, Artif. Intell..

[70]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.