MAP estimation in Binary MRFs via Bipartite Multi-cuts

We propose a new LP relaxation for obtaining the MAP assignment of a binary MRF with pairwise potentials. Our relaxation is derived from reducing the MAP assignment problem to an instance of a recently proposed Bipartite Multi-cut problem where the LP relaxation is guaranteed to provide an O(log k) approximation where k is the number of vertices adjacent to non-submodular edges in the MRF. We then propose a combinatorial algorithm to efficiently solve the LP and also provide a lower bound by concurrently solving its dual to within an e approximation. The algorithm is up to an order of magnitude faster and provides better MAP scores and bounds than the state of the art message passing algorithm of [1] that tightens the local marginal polytope with third-order marginal constraints.

[1]  Jochen Könemann,et al.  Faster and simpler algorithms for multicommodity flow and other fractional packing problems , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[2]  Tommi S. Jaakkola,et al.  Tree Block Coordinate Descent for MAP in Graphical Models , 2009, AISTATS.

[3]  Endre Boros,et al.  Pseudo-Boolean optimization , 2002, Discret. Appl. Math..

[4]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Vladimir Kolmogorov,et al.  Analyzing Convex Relaxations for MAP Estimation , 2009 .

[6]  Tomás Werner,et al.  High-arity interactions, polyhedral relaxations, and cutting plane algorithm for soft constraint optimisation (MAP-MRF) , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[8]  Sundar Vishwanathan,et al.  Approximation algorithms for the Bipartite Multicut problem , 2006, Inf. Process. Lett..

[9]  Éva Tardos,et al.  Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[10]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Nic Schraudolph,et al.  Polynomial-Time Exact Inference in NP-Hard Binary MRFs via Reweighted Perfect Matching , 2010, AISTATS.

[12]  Martin J. Wainwright,et al.  Message-passing for Graph-structured Linear Programs: Proximal Methods and Rounding Schemes , 2010, J. Mach. Learn. Res..

[13]  Nikos Komodakis,et al.  Beyond Loose LP-Relaxations: Optimizing MRFs by Repairing Cycles , 2008, ECCV.

[14]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[15]  Ali Ridha Mahjoub,et al.  On the cut polytope , 1986, Math. Program..

[16]  Tsuhan Chen,et al.  Beyond trees: MRF inference via outer-planar decomposition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Tommi S. Jaakkola,et al.  Approximate inference using planar graph decomposition , 2006, NIPS.

[18]  Martin J. Wainwright,et al.  Message-passing for graph-structured linear programs: proximal projections, convergence and rounding schemes , 2008, ICML '08.

[19]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.

[20]  Tommi S. Jaakkola,et al.  Tightening LP Relaxations for MAP using Message Passing , 2008, UAI.

[21]  Lisa Fleischer,et al.  Approximating fractional multicommodity flow independent of the number of commodities , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[22]  Mihalis Yannakakis,et al.  Approximate Max-Flow Min-(Multi)Cut Theorems and Their Applications , 1996, SIAM J. Comput..

[23]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[24]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Kyomin Jung,et al.  Local Rules for Global MAP: When Do They Work ? , 2009, NIPS.

[26]  Joseph Naor,et al.  Approximation algorithms for the metric labeling problem via a new linear programming formulation , 2001, SODA '01.

[27]  Uri Zwick,et al.  Outward rotations: a tool for rounding solutions of semidefinite programming relaxations, with applications to MAX CUT and other problems , 1999, STOC '99.

[28]  David Sontag,et al.  Cutting plane algorithms for variational inference in graphical models , 2007 .