Worst-case bounds on the quality of max-product fixed-points

We study worst-case bounds on the quality of any fixed point assignment of the max-product algorithm for Markov Random Fields (MRF). We start providing a bound independent of the MRF structure and parameters. Afterwards, we show how this bound can be improved for MRFs with specific structures such as bipartite graphs or grids. Our results provide interesting insight into the behavior of max-product. For example, we prove that max-product provides very good results (at least 90% optimal) on MRFs with large variable-disjoint cycles1.

[1]  Yair Weiss,et al.  Approximate Inference and Protein-Folding , 2002, NIPS.

[2]  Milind Tambe,et al.  Local optimization in cooperative agent networks , 2007 .

[3]  Bert Huang,et al.  Loopy Belief Propagation for Bipartite Maximum Weight b-Matching , 2007, AISTATS.

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[6]  Nicholas R. Jennings,et al.  Decentralised coordination of low-power embedded devices using the max-sum algorithm , 2008, AAMAS.

[7]  Tony Jebara,et al.  MAP Estimation, Message Passing, and Perfect Graphs , 2009, UAI.

[8]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Martin J. Wainwright,et al.  Tree consistency and bounds on the performance of the max-product algorithm and its generalizations , 2004, Stat. Comput..

[10]  Brendan J. Frey,et al.  Very loopy belief propagation for unwrapping phase images , 2001, NIPS.

[11]  Devavrat Shah,et al.  Message Passing for Maximum Weight Independent Set , 2008, IEEE Transactions on Information Theory.

[12]  Solomon Eyal Shimony,et al.  Finding MAPs for Belief Networks is NP-Hard , 1994, Artif. Intell..

[13]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[14]  Christian Borgs,et al.  Belief Propagation for Weighted b-Matchings on Arbitrary Graphs and its Relation to Linear Programs with Integer Solutions , 2007, SIAM J. Discret. Math..

[15]  Dmitry M. Malioutov,et al.  Linear programming analysis of loopy belief propagation for weighted matching , 2007, NIPS.

[16]  Milind Tambe,et al.  Quality Guarantees on k-Optimal Solutions for Distributed Constraint Optimization Problems , 2007, IJCAI.

[17]  Robert J. McEliece,et al.  The generalized distributive law , 2000, IEEE Trans. Inf. Theory.

[18]  Robert J. McEliece,et al.  Iterative min-sum decoding of tail-biting codes , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[19]  Brendan J. Frey,et al.  Introduction to the special issue on codes on graphs and iterative algorithms , 2001, IEEE Trans. Inf. Theory.

[20]  Milind Tambe,et al.  Asynchronous algorithms for approximate distributed constraint optimization with quality bounds , 2010, AAMAS.

[21]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[22]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.