Convergent Decomposition Solvers for Tree-reweighted Free Energies

We investigate minimization of treereweighted free energies for the purpose of obtaining approximate marginal probabilities and upper bounds on the partition function of cyclic graphical models. The solvers we present for this problem work by directly tightening tree-reweighted upper bounds. As a result, they are particularly ecient for tree-reweighted energies arising from a small number of spanning trees. While this assumption may seem restrictive at rst, we show how small sets of trees can be constructed in a principled manner. An appealing property of our algorithms, which results from the problem decomposition, is that they are embarrassingly parallel. In contrast to the original message passing algorithm introduced for this problem, we obtain global convergence guarantees.

[1]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[2]  Angelia Nedic,et al.  Approximately optimal utility maximization , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[3]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[4]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[5]  Mark W. Schmidt,et al.  Optimizing Costly Functions with Simple Constraints: A Limited-Memory Projected Quasi-Newton Algorithm , 2009, AISTATS.

[6]  Joseph Gonzalez,et al.  Residual Splash for Optimally Parallelizing Belief Propagation , 2009, AISTATS.

[7]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[8]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[9]  Martin J. Wainwright,et al.  A new class of upper bounds on the log partition function , 2002, IEEE Transactions on Information Theory.

[10]  Venkat Chandrasekaran,et al.  Complexity of Inference in Graphical Models , 2008, UAI.

[11]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[12]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[13]  J. Nocedal Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[14]  Jack Edmonds,et al.  Matroids and the greedy algorithm , 1971, Math. Program..

[15]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[16]  Tamir Hazan,et al.  Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies , 2008, UAI.

[17]  Nikos Komodakis,et al.  MRF Optimization via Dual Decomposition: Message-Passing Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Tommi S. Jaakkola,et al.  Convergent Propagation Algorithms via Oriented Trees , 2007, UAI.

[19]  Joris M. Mooij,et al.  libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models , 2010, J. Mach. Learn. Res..

[20]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[21]  Stephen Gould,et al.  Accelerated dual decomposition for MAP inference , 2010, ICML.

[22]  Changyu Wang,et al.  Convergence properties of nonmonotone spectral projected gradient methods , 2005 .

[23]  Tom Heskes,et al.  Convexity Arguments for Efficient Minimization of the Bethe and Kikuchi Free Energies , 2006, J. Artif. Intell. Res..