Combining Clauses with Various Precisions and Recalls to Produce Accurate Probabilistic Estimates

Statistical Relational Learning (SRL) combines the benefits of probabilistic machine learning approaches with complex, structured domains from Inductive Logic Programming (ILP). We propose a new SRL algorithm, GleanerSRL, to generate the probability that an example is positive within highly-skewed relational domains. In this work, we combine clauses from Gleaner, an ILP algorithm for learning a wide variety of first-order clauses, with the propositional learning technique of support vector machines to learn well-calibrated probabilities. We find that our results are comparable to SRL algorithms SAYU and SAYUVISTA on a well-known relational testbed.

[1]  Igor Kononenko,et al.  Naive Bayesian classifier within ILP-R , 1995 .

[2]  Ashwin Srinivasan,et al.  Feature Construction with Inductive Logic Programming: A Study of Quantitative Predictions of Biological Activity by Structural Attributes , 1996, Inductive Logic Programming Workshop.

[3]  Avi Pfeffer,et al.  Learning Probabilities for Noisy First-Order Rules , 1997, IJCAI.

[4]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[5]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[6]  Mark Craven,et al.  Representing Sentence Structure in Hidden Markov Models for Information Extraction , 2001, IJCAI.

[7]  Nada Lavrač,et al.  An Introduction to Inductive Logic Programming , 2001 .

[8]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[9]  Ben Taskar,et al.  Link Prediction in Relational Data , 2003, NIPS.

[10]  P. Abbeel,et al.  Label and Link Prediction in Relational Data , 2003 .

[11]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[12]  Ashwin Srinivasan,et al.  Feature construction with Inductive Logic Programming: A Study of Quantitative Predictions of Biological Activity Aided by Structural Attributes , 1999, Data Mining and Knowledge Discovery.

[13]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[14]  Jesse Davis,et al.  Establishing Identity Equivalence in Multi-Relational Domains , 2005 .

[15]  Stephen Muggleton,et al.  Support Vector Inductive Logic Programming , 2005, Discovery Science.

[16]  Rich Caruana,et al.  Predicting good probabilities with supervised learning , 2005, ICML.

[17]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[18]  Luc De Raedt,et al.  kFOIL: Learning Simple Relational Kernels , 2006, AAAI.

[19]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[20]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[21]  Jude W. Shavlik,et al.  Gleaner: Creating ensembles of first-order clauses to improve recall-precision curves , 2006, Machine Learning.

[22]  Jesse Davis,et al.  Change of Representation for Statistical Relational Learning , 2007, IJCAI.

[23]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[24]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .