Automorphism Groups of Graphical Models and Lifted Variational Inference

Using the theory of group action, we first introduce the concept of the automorphism group of an exponential family or a graphical model, thus formalizing the general notion of symmetry of a probabilistic model. This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family. Its group action partitions the set of random variables and feature functions into equivalent classes (called orbits) having identical marginals and expectations. Then the inference problem is effectively reduced to that of computing marginals or expectations for each class, thus avoiding the need to deal with each individual variable or feature. We demonstrate the usefulness of this general framework in lifting two classes of variational approximation for MAP inference: local LP relaxation and local LP relaxation with cycle constraints; the latter yields the first lifted inference that operate on a bound tighter than local constraints. Initial experimental results demonstrate that lifted MAP inference with cycle constraints achieved the state of the art performance, obtaining much better objective function values than local approximation while remaining relatively efficient.

[1]  Ofer Meshi,et al.  Template Based Inference in Symmetric Relational Markov Random Fields , 2007, UAI.

[2]  W. Ben‐Ameur,et al.  Acceleration of cutting-plane and column generation algorithms: Applications to network design , 2007 .

[3]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[4]  Mathias Niepert Lifted Probabilistic Inference: An MCMC Perspective , 2012, StarAI@UAI.

[5]  Gordon F. Royle,et al.  Algebraic Graph Theory , 2001, Graduate texts in mathematics.

[6]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[7]  Tuyen N. Huynh,et al.  Exact Lifted Inference with Distinct Soft Evidence on Every Object , 2012, AAAI.

[8]  Kristian Kersting,et al.  Lifted Linear Programming , 2012, AISTATS.

[9]  Ove Frank,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[10]  Pedro M. Domingos,et al.  Exploiting Logical Structure in Lifted Probabilistic Inference , 2010, StarAI@AAAI.

[11]  Pedro M. Domingos,et al.  Lifted First-Order Belief Propagation , 2008, AAAI.

[12]  Hanif D. Sherali,et al.  A Hierarchy of Relaxations Between the Continuous and Convex Hull Representations for Zero-One Programming Problems , 1990, SIAM J. Discret. Math..

[13]  J. Yeh,et al.  Real Analysis: Theory of Measure and Integration , 2006 .

[14]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[15]  Warren P. Adams,et al.  A hierarchy of relaxation between the continuous and convex hull representations , 1990 .

[16]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[17]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[18]  Tommi S. Jaakkola,et al.  Approximate inference in graphical models using lp relaxations , 2010 .

[19]  Brendan D. McKay,et al.  Practical graph isomorphism, II , 2013, J. Symb. Comput..

[20]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.