Lifted Probabilistic Inference

Many AI problems arising in a wide variety of fields such as machine learning, semantic web, network communication, computer vision, and robotics can elegantly be encoded and solved using probabilistic graphical models. Often, however, we are facing inference problems with symmetries and redundancies only implicitly captured in the graph structure and, hence, not exploitable by efficient inference approaches. A prominent example are probabilistic logical models that tackle a long standing goal of AI, namely unifying first-order logic — capturing regularities and symmetries — and probability — capturing uncertainty. Although they often encode large, complex models using few rules only and, hence, symmetries and redundancies abound, inference in them was originally still at the propositional representation level and did not exploit symmetries. This paper is intended to give a (not necessarily complete) overview and invitation to the emerging field of lifted probabilistic inference, inference techniques that exploit these symmetries in graphical models in order to speed up inference, ultimately orders of magnitude.

[1]  Ofer Meshi,et al.  Template Based Inference in Symmetric Relational Markov Random Fields , 2007, UAI.

[2]  Guy Van den Broeck,et al.  Conditioning in First-Order Knowledge Compilation and Lifted Probabilistic Inference , 2012, AAAI.

[3]  Dan Suciu,et al.  Lifted Inference Seen from the Other Side : The Tractable Features , 2010, NIPS.

[4]  John Stillwell,et al.  Symmetry , 2000, Am. Math. Mon..

[5]  Pedro M. Domingos,et al.  Probabilistic theorem proving , 2011, UAI.

[6]  Michael Joswig,et al.  Algorithms for highly symmetric linear and integer programs , 2010, Mathematical Programming.

[7]  Kristian Kersting,et al.  Markov Logic Sets: Towards Lifted Information Retrieval Using PageRank and Label Propagation , 2011, AAAI.

[8]  Helene Gehrmann,et al.  Lattices of Graphical Gaussian Models with Symmetries , 2011, Symmetry.

[9]  Fahiem Bacchus,et al.  Towards Completely Lifted Search-based Probabilistic Inference , 2011, ArXiv.

[10]  Toby Walsh,et al.  Symmetry Breaking Constraints: Recent Results , 2012, AAAI.

[11]  Kristian Kersting,et al.  Lifted Online Training of Relational Models with Stochastic Gradient Methods , 2012, ECML/PKDD.

[12]  Fabian Hadiji,et al.  Lifted Message Passing for Satisfiability , 2010, Statistical Relational Artificial Intelligence.

[13]  Leslie Pack Kaelbling,et al.  Logical Particle Filtering , 2007, Probabilistic, Logical and Relational Learning - A Further Synthesis.

[14]  François Margot,et al.  Symmetry in Integer Linear Programming , 2010, 50 Years of Integer Programming.

[15]  Jaesik Choi,et al.  Lifted Relational Kalman Filtering , 2011, IJCAI.

[16]  P. Erdos,et al.  Asymmetric graphs , 1963 .

[17]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[18]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[19]  Scott Sanner,et al.  Multi-Evidence Lifted Message Passing, with Application to PageRank and the Kalman Filter , 2011, IJCAI.

[20]  Pedro M. Domingos,et al.  Efficient Lifting for Online Probabilistic Inference , 2010, AAAI.

[21]  Balaraman Ravindran,et al.  Symmetries and Model Minimization in Markov Decision Processes , 2001 .

[22]  Roni Khardon,et al.  Stochastic Planning and Lifted Inference , 2017, StarAI@AAAI.

[23]  Sriraam Natarajan,et al.  Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network , 2009, IJCAI.

[24]  Dahlia W. Zaidel,et al.  Asymmetry and Symmetry in the Beauty of Human Faces , 2010, Symmetry.

[25]  K. Kersting,et al.  Lifted Belief Propagation : Pairwise Marginals and Beyond , 2010 .

[26]  Stuart J. Russell,et al.  General-Purpose MCMC Inference over Relational Structures , 2006, UAI.

[27]  Jesse Davis,et al.  Lifted Variable Elimination with Arbitrary Constraints , 2012, AISTATS.

[28]  David Poole,et al.  Lifted Aggregation in Directed First-Order Probabilistic Models , 2009, IJCAI.

[29]  Adnan Darwiche,et al.  Recursive conditioning , 2001, Artif. Intell..

[30]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[31]  David J. Hill,et al.  Lifted Inference for Relational Continuous Models , 2010, Statistical Relational Artificial Intelligence.

[32]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[33]  David Poole,et al.  Constraint Processing in Lifted Probabilistic Inference , 2009, UAI.

[34]  Dan Klein,et al.  Type-Based MCMC , 2010, HLT-NAACL.

[35]  Luc De Raedt,et al.  Lifted Probabilistic Inference by First-Order Knowledge Compilation , 2011, IJCAI.

[36]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[37]  Adnan Darwiche,et al.  Relax, Compensate and Then Recover , 2010, JSAI-isAI Workshops.

[38]  Steffen L. Lauritzen,et al.  Estimation of means in graphical Gaussian models with symmetries , 2011, 1101.3709.

[39]  Matthew Richardson,et al.  Speeding Up Inference in Statistical Relational Learning by Clustering Similar Query Literals , 2009, ILP.

[40]  Fabian Hadiji,et al.  Efficient Sequential Clamping for Lifted Message Passing , 2011, KI.

[41]  Nils J. Nilsson,et al.  Probabilistic Logic * , 2022 .

[42]  Pedro M. Domingos,et al.  Lifted First-Order Belief Propagation , 2008, AAAI.

[43]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[44]  Tuyen N. Huynh,et al.  Exact Lifted Inference with Distinct Soft Evidence on Every Object , 2012, AAAI.

[45]  Matthias Thimm,et al.  On Prototypical Indifference and Lifted Inference in Relational Probabilistic Conditional Logic , 2012 .

[46]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming , 2004, Probabilistic Inductive Logic Programming.

[47]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[48]  Scott Sanner,et al.  Multi-evidence Lifted Message Passing , 2011 .

[49]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[50]  Scott Sanner,et al.  Practical solution techniques for first-order MDPs , 2009, Artif. Intell..

[51]  Ian Stewart,et al.  Why Beauty Is Truth: A History of Symmetry , 2007 .

[52]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[53]  Leslie Pack Kaelbling,et al.  Lifted Probabilistic Inference with Counting Formulas , 2008, AAAI.

[54]  Dan Roth,et al.  MPE and Partial Inversion in Lifted Probabilistic Variable Elimination , 2006, AAAI.

[55]  Pedro M. Domingos,et al.  A General Method for Reducing the Complexity of Relational Inference and its Application to MCMC , 2008, AAAI.

[56]  Pedro M. Domingos,et al.  Approximate Lifted Belief Propagation , 2010, StarAI@AAAI.

[57]  Kristian Kersting,et al.  Lifted Linear Programming , 2012, AISTATS.

[58]  R. Bodi,et al.  SYMMETRIES IN LINEAR AND INTEGER PROGRAMS , 2009, 0908.3329.

[59]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on Graphs with Symmetries , 2007, SIAM J. Optim..

[60]  Luc De Raedt,et al.  Stochastic relational processes: Efficient inference and applications , 2011, Machine Learning.

[61]  Lise Getoor,et al.  Bisimulation-based Approximate Lifted Inference , 2009, UAI.

[62]  Jaesik Choi,et al.  Efficient Methods for Lifted Inference with Aggregate Factors , 2011, AAAI.

[63]  Guy Van den Broeck On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference , 2011, NIPS.

[64]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[65]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[66]  Sebastian Riedel Improving the Accuracy and Efficiency of MAP Inference for Markov Logic , 2008, UAI.

[67]  Pedro M. Domingos,et al.  Coarse-to-Fine Inference and Learning for First-Order Probabilistic Models , 2011, AAAI.

[68]  Fabian Hadiji,et al.  Informed Lifting for Message-Passing , 2010, AAAI.

[69]  Lise Getoor,et al.  Exploiting shared correlations in probabilistic databases , 2008, Proc. VLDB Endow..

[70]  Pascal Van Hentenryck,et al.  Structural Symmetry Breaking , 2005, IJCAI.

[71]  Robert Givan,et al.  Model Minimization in Markov Decision Processes , 1997, AAAI/IAAI.

[72]  Luc De Raedt,et al.  Logical and Relational Learning: From ILP to MRDM (Cognitive Technologies) , 2008 .

[73]  Guy Van den Broeck,et al.  Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference , 2012, UAI.

[74]  Pedro M. Domingos,et al.  Exploiting Logical Structure in Lifted Probabilistic Inference , 2010, StarAI@AAAI.

[75]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.