On the Complexity and Approximation of Binary Evidence in Lifted Inference

Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relational models, but often resort to non-lifted inference when computing conditional probabilities, because the evidence breaks many of the model's symmetries. Recent theoretical results paint a grim picture, showing that conditioning on binary relations is #P-hard, and in the worst case, no lifting can be expected. In this paper, we identify Boolean rank of the evidence as a key parameter in the complexity of conditioning. We contrast the hardness result by showing that conditioning on binary evidence with bounded Boolean rank is efficient. This opens up the possibility of approximating evidence by a low-rank Boolean matrix factorization that maintains the model's symmetries and admits efficient lifted inference.

[1]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[2]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[3]  Mathias Niepert,et al.  Symmetry-Aware Marginal Density Estimation , 2013, AAAI.

[4]  Pauli Miettinen,et al.  Model order selection for boolean matrix factorization , 2011, KDD.

[5]  Pauli Miettinen Sparse Boolean Matrix Factorizations , 2010, 2010 IEEE International Conference on Data Mining.

[6]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[7]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[8]  Guy Van den Broeck,et al.  Conditioning in First-Order Knowledge Compilation and Lifted Probabilistic Inference , 2012, AAAI.

[9]  Hans L. Bodlaender,et al.  Treewidth: Algorithmic Techniques and Results , 1997, MFCS.

[10]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[11]  Guy Van den Broeck,et al.  Liftability of Probabilistic Inference: Upper and Lower Bounds , 2012 .

[12]  Dan Suciu,et al.  Lifted Inference Seen from the Other Side : The Tractable Features , 2010, NIPS.

[13]  Fabian Hadiji,et al.  Informed Lifting for Message-Passing , 2010, AAAI.

[14]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming - Theory and Applications , 2008, Probabilistic Inductive Logic Programming.

[15]  Tuyen N. Huynh,et al.  Exact Lifted Inference with Distinct Soft Evidence on Every Object , 2012, AAAI.

[16]  Bart Goethals,et al.  Tiling Databases , 2004, Discovery Science.

[17]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[18]  Jesse Davis,et al.  Lifted Variable Elimination with Arbitrary Constraints , 2012, AISTATS.

[19]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[20]  Luc De Raedt,et al.  Lifted Probabilistic Inference by First-Order Knowledge Compilation , 2011, IJCAI.

[21]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[22]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[23]  Guy Van den Broeck Lifted Inference and Learning in Statistical Relational Models , 2013 .

[24]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[25]  Pauli Miettinen,et al.  The Discrete Basis Problem , 2006, IEEE Transactions on Knowledge and Data Engineering.

[26]  Mark Craven,et al.  Relational Learning with Statistical Predicate Invention: Better Models for Hypertext , 2001, Machine Learning.

[27]  Václav Snásel,et al.  Developing Genetic Algorithms for Boolean Matrix Factorization , 2008, DATESO.

[28]  Guy Van den Broeck,et al.  Lifted Generative Parameter Learning , 2013, StarAI@AAAI.

[29]  Guy Van den Broeck,et al.  Completeness Results for Lifted Variable Elimination , 2013, AISTATS.

[30]  Guy Van den Broeck On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference , 2011, NIPS.

[31]  Pedro M. Domingos,et al.  Deep transfer via second-order Markov logic , 2009, ICML '09.

[32]  Pedro M. Domingos,et al.  Probabilistic theorem proving , 2011, UAI.

[33]  Leslie Pack Kaelbling,et al.  Lifted Probabilistic Inference with Counting Formulas , 2008, AAAI.

[34]  Pauli Miettinen,et al.  Matrix Decomposition Methods for Data Mining : Computational Complexity and Algorithms , 2009 .

[35]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .