New Liftable Classes for First-Order Probabilistic Inference

Statistical relational models provide compact encodings of probabilistic dependencies in relational domains, but result in highly intractable graphical models. The goal of lifted inference is to carry out probabilistic inference without needing to reason about each individual separately, by instead treating exchangeable, undistinguished objects as a whole. In this paper, we study the domain recursion inference rule, which, despite its central role in early theoretical results on domain-lifted inference, has later been believed redundant. We show that this rule is more powerful than expected, and in fact significantly extends the range of models for which lifted inference runs in time polynomial in the number of individuals in the domain. This includes an open problem called S4, the symmetric transitivity model, and a first-order logic encoding of the birthday paradox. We further identify new classes S2FO2 and S2RU of domain-liftable theories, which respectively subsume FO2 and recursively unary theories, the largest classes of domain-liftable theories known so far, and show that using domain recursion can achieve exponential speedup even in theories that cannot fully be lifted with the existing set of inference rules.

[1]  Guy Van den Broeck,et al.  Completeness Results for Lifted Variable Elimination , 2013, AISTATS.

[2]  Jaesik Choi,et al.  Efficient Methods for Lifted Inference with Aggregate Factors , 2011, AAAI.

[3]  Guy Van den Broeck.  On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference , 2011, NIPS.

[4]  Vibhav Gogate,et al.  Evidence-Based Clustering for Scalable Inference in Markov Logic , 2014, ECML/PKDD.

[5]  Dan Suciu,et al.  Lifted Inference Seen from the Other Side : The Tractable Features , 2010, NIPS.

[6]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[7]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[8]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[9]  Luc De Raedt,et al.  Lifted Probabilistic Inference by First-Order Knowledge Compilation , 2011, IJCAI.

[10]  Prasoon Goyal,et al.  Probabilistic Databases , 2009, Encyclopedia of Database Systems.

[11]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[12]  Guy Van den Broeck,et al.  Symmetric Weighted First-Order Model Counting , 2014, PODS.

[13]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[14]  Guy Van den Broeck,et al.  Skolemization for Weighted First-Order Model Counting , 2013, KR.

[15]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[16]  Hung Hai Bui,et al.  Automorphism Groups of Graphical Models and Lifted Variational Inference , 2012, UAI.

[17]  Seyed Mehran Kazemi,et al.  Knowledge Compilation for Lifted Probabilistic Inference: Compiling to a Low-Level Language , 2016, KR.

[18]  Alexander M. Rush,et al.  A Fast Variational Approach for Learning Markov Random Field Language Models , 2015, ICML.

[19]  Fahiem Bacchus,et al.  Towards Completely Lifted Search-based Probabilistic Inference , 2011, ArXiv.

[20]  Kristian Kersting,et al.  Lifted Online Training of Relational Models with Stochastic Gradient Methods , 2012, ECML/PKDD.

[21]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[22]  Guy Van den Broeck,et al.  Lifted generative learning of Markov logic networks , 2016, Machine Learning.

[23]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[24]  Vibhav Gogate,et al.  Scaling-up Importance Sampling for Markov Logic Networks , 2014, NIPS.

[25]  Dan Suciu,et al.  Efficient query evaluation on probabilistic databases , 2004, The VLDB Journal.

[26]  Pedro M. Domingos,et al.  Probabilistic theorem proving , 2011, UAI.

[27]  Leslie Pack Kaelbling,et al.  Lifted Probabilistic Inference with Counting Formulas , 2008, AAAI.

[28]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[29]  Guy Van den Broeck,et al.  Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference , 2012, UAI.

[30]  Henry A. Kautz,et al.  Lifted Symmetry Detection and Breaking for MAP Inference , 2015, NIPS.

[31]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[32]  Pedro M. Domingos,et al.  Lifted First-Order Belief Propagation , 2008, AAAI.

[33]  Seyed Mehran Kazemi,et al.  Why is Compiling Lifted Inference into a Low-Level Language so Effective? , 2016, ArXiv.