Block-Value Symmetries in Probabilistic Graphical Models

One popular way for lifted inference in probabilistic graphical models is to first merge symmetric states into a single cluster (orbit) and then use these for downstream inference, via variations of orbital MCMC [Niepert, 2012]. These orbits are represented compactly using permutations over variables, and variable-value (VV) pairs, but they can miss several state symmetries in a domain. We define the notion of permutations over block-value (BV) pairs, where a block is a set of variables. BV strictly generalizes VV symmetries, and can compute many more symmetries for increasing block sizes. To operationalize use of BV permutations in lifted inference, we describe 1) an algorithm to compute BV permutations given a block partition of the variables, 2) BV-MCMC, an extension of orbital MCMC that can sample from BV orbits, and 3) a heuristic to suggest good block partitions. Our experiments show that BV-MCMC can mix much faster compared to vanilla MCMC and orbital MCMC.

[1]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[2]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[3]  Parag Singla,et al.  Contextual Symmetries in Probabilistic Graphical Models , 2016, IJCAI.

[4]  Lise Getoor,et al.  Lifted graphical models: a survey , 2011, Machine Learning.

[5]  Scott H. Murray,et al.  Generating random elements of a finite group , 1995 .

[6]  Parag Singla,et al.  Coarse-to-Fine Lifted MAP Inference in Computer Vision , 2017, IJCAI.

[7]  Guy Van den Broeck,et al.  Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference , 2014, AAAI.

[8]  Pedro M. Domingos,et al.  Markov Logic: An Interface Layer for Artificial Intelligence , 2009, Markov Logic: An Interface Layer for Artificial Intelligence.

[9]  Guy Van den Broeck,et al.  Lifted Probabilistic Inference for Asymmetric Graphical Models , 2014, AAAI.

[10]  Igor L. Markov,et al.  Faster symmetry discovery using sparsity of symmetries , 2008, 2008 45th ACM/IEEE Design Automation Conference.

[11]  Kristian Kersting,et al.  Lifted Linear Programming , 2012, AISTATS.

[12]  Hung Hai Bui,et al.  Automorphism Groups of Graphical Models and Lifted Variational Inference , 2012, UAI.

[13]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[14]  Parag Singla,et al.  Non-Count Symmetries in Boolean & Multi-Valued Prob. Graphical Models , 2017, AISTATS.

[15]  Igor Pak,et al.  The product replacement algorithm is polynomial , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[16]  Vibhav Gogate,et al.  On Lifting the Gibbs Sampling Algorithm , 2012, StarAI@UAI.

[17]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[18]  Pedro M. Domingos,et al.  Lifted First-Order Belief Propagation , 2008, AAAI.

[19]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.