Reference classes and relational learning

This paper studies the connections between relational probabilistic models and reference classes, with specific focus on the ability of these models to generate the correct answers to probabilistic queries. We distinguish between relational models that represent only observed relations and those which additionally represent latent properties of individuals. We show how both types of relational models can be understood in terms of reference classes, and that learning such models correspond to different ways of identifying reference classes. Rather than examining the impact of philosophical issues associated with reference classes on relational learning, we directly assess whether relational models can represent the correct probabilities of a simple generative process for relational data. We show that models with only observed properties and relations can only represent the correct probabilities under restrictive conditions, whilst models that also represent latent properties avoids such restrictions. As such, methods for acquiring latent-property models are an attractive alternatives to traditional ways of identifying reference classes. Our experiments on synthetic as well as real-world domains support the analysis, demonstrating that models with latent relations are significantly more accurate than those without latent relations.

[1]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[2]  Ben Taskar,et al.  Probabilistic Classification and Clustering in Relational Data , 2001, IJCAI.

[3]  Hans Reichenbach,et al.  The theory of probability , 1968 .

[4]  J. R. Quinlan Learning Logical Definitions from Relations , 1990 .

[5]  Nevin Lianwen Zhang,et al.  Exploiting Causal Independence in Bayesian Network Inference , 1996, J. Artif. Intell. Res..

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[7]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[8]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[9]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[10]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[11]  Richard S. Zemel,et al.  The multiple multiplicative factor model for collaborative filtering , 2004, ICML.

[12]  Thomas Hofmann,et al.  Latent Class Models for Collaborative Filtering , 1999, IJCAI.

[13]  Yi Shen,et al.  Loss functions for binary classification and class probability estimation , 2005 .

[14]  James H. Fetzer Reichenbach, reference classes, and single case ‘probabilities’ , 1977, Synthese.

[15]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[16]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[17]  Henry E. Kyburg,,et al.  The Reference Class , 1983, Philosophy of Science.

[18]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[19]  Joseph Y. Halpern An Analysis of First-Order Logics of Probability , 1989, IJCAI.

[20]  Joseph Y. Halpern,et al.  From Statistical Knowledge Bases to Degrees of Belief , 1996, Artif. Intell..

[21]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[22]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[23]  Dean P. Foster,et al.  Clustering Methods for Collaborative Filtering , 1998, AAAI 1998.

[24]  Henry Ely Kyburg,et al.  The logical foundations of statistical inference , 1974 .

[25]  S. Muggleton Stochastic Logic Programs , 1996 .

[26]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[27]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[28]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[29]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[30]  Timothy J. McGrew Direct Inference and the Problem of Induction , 2001 .

[31]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[32]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[33]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[34]  David Poole,et al.  The Independent Choice Logic for Modelling Multiple Agents Under Uncertainty , 1997, Artif. Intell..

[35]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[36]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[37]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[38]  Luc De Raedt,et al.  Logical and Relational Learning: From ILP to MRDM (Cognitive Technologies) , 2008 .

[39]  Kyburg,et al.  Randomness and the Right Reference Class , 1977 .

[40]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[41]  Thomas L. Griffiths,et al.  Learning Systems of Concepts with an Infinite Relational Model , 2006, AAAI.

[42]  Chris H Wiggins,et al.  Bayesian approach to network modularity. , 2007, Physical review letters.

[43]  A. Raftery,et al.  Model‐based clustering for social networks , 2007 .

[44]  I. Levi Direct Inference and Confirmational Conditionalization , 1981, Philosophy of Science.

[45]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[46]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[47]  Hans-Peter Kriegel,et al.  Infinite Hidden Relational Models , 2006, UAI.

[48]  Kristian Kersting,et al.  Social Network Mining with Nonparametric Relational Models , 2008, SNAKDD.

[49]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[50]  Henry E. Kyburg,et al.  Believing on the Basis of the Evidence , 2007, Comput. Intell..

[51]  Edoardo M. Airoldi,et al.  Mixed Membership Stochastic Blockmodels , 2007, NIPS.