Lifted Probabilistic Inference for Asymmetric Graphical Models

Lifted probabilistic inference algorithms have been successfully applied to a large number of symmetric graphical models. Unfortunately, the majority of real-world graphical models is asymmetric. This is even the case for relational representations when evidence is given. Therefore, more recent work in the community moved to making the models symmetric and then applying existing lifted inference algorithms. However, this approach has two shortcomings. First, all existing over-symmetric approximations require a relational representation such as Markov logic networks. Second, the induced symmetries often change the distribution significantly, making the computed probabilities highly biased. We present a framework for probabilistic sampling-based inference that only uses the induced approximate symmetries to propose steps in a Metropolis-Hastings style Markov chain. The framework, therefore, leads to improved probability estimates while remaining unbiased. Experiments demonstrate that the approach outperforms existing MCMC algorithms.

[1]  Tuyen N. Huynh,et al.  Exact Lifted Inference with Distinct Soft Evidence on Every Object , 2012, AAAI.

[2]  Mathias Niepert,et al.  Symmetry-Aware Marginal Density Estimation , 2013, AAAI.

[3]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming - Theory and Applications , 2008, Probabilistic Inductive Logic Programming.

[4]  Kristian Kersting,et al.  Lifted Inference via k-Locality , 2013, StarAI@AAAI.

[5]  Heiner Stuckenschmidt,et al.  RockIt: Exploiting Parallelism and Symmetry for MAP Inference in Statistical Relational Models , 2013, AAAI.

[6]  Vibhav Gogate,et al.  On Lifting the Gibbs Sampling Algorithm , 2012, StarAI@UAI.

[7]  Roman Garnett,et al.  Power Iterated Color Refinement , 2014, AAAI.

[8]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[9]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[10]  Hung Hai Bui,et al.  Automorphism Groups of Graphical Models and Lifted Variational Inference , 2012, UAI.

[11]  Kristian Kersting,et al.  Lifted Probabilistic Inference , 2012, ECAI.

[12]  Guy Van den Broeck,et al.  Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference , 2014, AAAI.

[13]  Mathias Niepert Lifted Probabilistic Inference: An MCMC Perspective , 2012, StarAI@UAI.

[14]  Vibhav Gogate,et al.  Evidence-Based Clustering for Scalable Inference in Markov Logic , 2014, ECML/PKDD.

[15]  P. Erdös ASYMMETRIC GRAPHS , 2022 .

[16]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[17]  Daniel A. Lidar,et al.  Evidence for quantum annealing with more than one hundred qubits , 2013, Nature Physics.

[18]  Guy Van den Broeck,et al.  Understanding the Complexity of Lifted Inference and Asymmetric Weighted Model Counting , 2014, StarAI@AAAI.

[19]  Guy Van den Broeck On the Complexity and Approximation of Binary Evidence in Lifted Inference , 2013, StarAI@AAAI.

[20]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[21]  Mark Craven,et al.  Relational Learning with Statistical Predicate Invention: Better Models for Hypertext , 2001, Machine Learning.

[22]  Olle Häggström Finite Markov Chains and Algorithmic Applications , 2002 .

[23]  Pedro M. Domingos,et al.  Approximate Lifting Techniques for Belief Propagation , 2014, AAAI.

[24]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[25]  Guy Van den Broeck,et al.  Liftability of Probabilistic Inference: Upper and Lower Bounds , 2012 .

[26]  Guy Van den Broeck Lifted Inference and Learning in Statistical Relational Models , 2013 .

[27]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[28]  Pedro M. Domingos,et al.  Sound and Efficient Inference with Probabilistic and Deterministic Dependencies , 2006, AAAI.

[29]  Linda C. van der Gaag,et al.  Probabilistic Graphical Models , 2014, Lecture Notes in Computer Science.

[30]  L. Tierney Markov Chains for Exploring Posterior Distributions , 1994 .

[31]  Guy Van den Broeck On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference , 2011, NIPS.

[32]  Vibhav Gogate,et al.  Advances in Lifted Importance Sampling , 2012, AAAI.

[33]  Adnan Darwiche,et al.  Modeling and Reasoning with Bayesian Networks , 2009 .