Generalized sequential tree-reweighted message passing

This paper addresses the problem of approximate MAP-MRF inference in general graphical models. Following [36], we consider a family of linear programming relaxations of the problem where each relaxation is specified by a set of nested pairs of factors for which the marginalization constraint needs to be enforced. We develop a generalization of the TRW-S algorithm [9] for this problem, where we use a decomposition into junction chains, monotonic w.r.t. some ordering on the nodes. This generalizes the monotonic chains in [9] in a natural way. We also show how to deal with nested factors in an efficient way. Experiments show an improvement over min-sum diffusion, MPLP and subgradient ascent algorithms on a number of computer vision and natural language processing problems.

[1]  Tamir Hazan,et al.  Norm-Product Belief Propagation: Primal-Dual Message-Passing for Approximate Inference , 2009, IEEE Transactions on Information Theory.

[2]  Richard S. Zemel,et al.  HOP-MAP: Efficient Message Passing with High Order Potentials , 2010, AISTATS.

[3]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[4]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[5]  Alexander M. Rush,et al.  Dual Decomposition for Parsing with Non-Projective Head Automata , 2010, EMNLP.

[6]  Tommi S. Jaakkola,et al.  Tightening LP Relaxations for MAP using Message Passing , 2008, UAI.

[7]  Sebastian Nowozin,et al.  Tighter Relaxations for MAP-MRF Inference: A Local Primal-Dual Gap based Separation Algorithm , 2011, AISTATS.

[8]  Stephen Gould,et al.  Accelerated dual decomposition for MAP inference , 2010, ICML.

[9]  Vladimir Kolmogorov,et al.  Joint optimization of segmentation and appearance models , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Daniel Cremers,et al.  A Linear Framework for Region-Based Image Segmentation and Inpainting Involving Curvature Penalization , 2011, International Journal of Computer Vision.

[11]  Philip H. S. Torr,et al.  Efficiently solving convex relaxations for MAP estimation , 2008, ICML '08.

[12]  Yuan Qi,et al.  Tree-structured Approximations by Expectation Propagation , 2003, NIPS.

[13]  Martin J. Wainwright,et al.  Message-passing for graph-structured linear programs: proximal projections, convergence and rounding schemes , 2008, ICML '08.

[14]  Thomas Schoenemann,et al.  Probabilistic Word Alignment under the L_0-norm , 2011, CoNLL.

[15]  Adnan Darwiche,et al.  Inference in belief networks: A procedural guide , 1996, Int. J. Approx. Reason..

[16]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[17]  Christoph Schnörr,et al.  Efficient MRF Energy Minimization via Adaptive Diminishing Smoothing , 2012, UAI.

[18]  Tommi S. Jaakkola,et al.  Approximate inference in graphical models using lp relaxations , 2010 .

[19]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Leo Grady,et al.  Fast global optimization of curvature , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Tai Sing Lee,et al.  Efficient belief propagation for higher-order cliques using linear constraint nodes , 2008, Comput. Vis. Image Underst..

[22]  Vladimir Kolmogorov,et al.  A Dual Decomposition Approach to Feature Correspondence , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nikos Komodakis,et al.  Beyond pairwise energies: Efficient optimization for higher-order MRFs , 2009, CVPR.

[24]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Fredrik Kahl,et al.  Curvature Regularization for Curves and Surfaces in a Global Optimization Framework , 2011, EMMCVPR.

[26]  Nikos Komodakis,et al.  MRF Optimization via Dual Decomposition: Message-Passing Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[27]  Eric P. Xing,et al.  An Augmented Lagrangian Approach to Constrained MAP Inference , 2011, ICML.

[28]  Yair Weiss,et al.  Linear Programming Relaxations and Belief Propagation - An Empirical Study , 2006, J. Mach. Learn. Res..

[29]  Tomás Werner,et al.  Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Nikos Komodakis,et al.  Beyond Loose LP-Relaxations: Optimizing MRFs by Repairing Cycles , 2008, ECCV.

[31]  Christoph Schnörr,et al.  A study of Nesterov's scheme for Lagrangian decomposition and MAP labeling , 2011, CVPR 2011.

[32]  Vladimir Kolmogorov,et al.  Dynamic Tree Block Coordinate Ascent , 2011, ICML.

[33]  D. Sontag.  1 Introduction to Dual Decomposition for Inference , 2010 .

[34]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[35]  Ofer Meshi,et al.  An Alternating Direction Method for Dual MAP LP Relaxation , 2011, ECML/PKDD.

[36]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.