Solving M-Modes in Loopy Graphs Using Tree Decompositions

M-Modes is the problem of finding the top M labelings of a graphical model that are locally optimal. The state-of-the-art M-Modes algorithm is a heuristic search method that finds global modes by incrementally concatenating MAP solutions in local neighborhoods. The search method also relies on the guidance of a heuristic function to explore the most promising parts of the search space. However, due to the difficulty of coordinating mode search, heuristic function calculation and local MAP computation in general loopy graphs, the method was only implemented and tested on special graphical models such as trees or submodular grid graphs. This paper provides a more general implementation of the search method based on tree decompositions that is applicable to general loopy graphs. A tree decomposition allows a sequence of local subgraphs to be mapped to a set of sub-trees sweeping through the tree decomposition, thus enabling a smooth and efficient transition back and forth between mode search, heuristic function calculation and local MAP calculations. We use both random and real datasets to evaluate the effectiveness of the tree-decomposition method. Furthermore, we demonstrate the practical value of M-Modes in making multiple diverse structured predictions for a gesture recognition task.

[1]  Paul D. Seymour,et al.  Graph minors. III. Planar tree-width , 1984, J. Comb. Theory, Ser. B.

[2]  Amir Globerson,et al.  An LP View of the M-best MAP problem , 2009, NIPS.

[3]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[4]  Rina Dechter,et al.  Search Algorithms for m Best Solutions for Graphical Models , 2012, AAAI.

[5]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[6]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[7]  Y. Weiss,et al.  Finding the M Most Probable Configurations using Loopy Belief Propagation , 2003, NIPS 2003.

[8]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[9]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[10]  Sebastian Nowozin,et al.  Structured Learning and Prediction in Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[11]  Carsten Rother,et al.  Inferring M-Best Diverse Labelings in a Single One , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Chao Chen,et al.  Mode Estimation for High Dimensional Discrete Tree Graphical Models , 2014, NIPS.

[13]  Christoph H. Lampert,et al.  Computing the M Most Probable Modes of a Graphical Model , 2013, AISTATS.

[14]  Christoph H. Lampert.  Maximum Margin Multi-Label Structured Prediction , 2011, NIPS.

[15]  Changhe Yuan,et al.  Solving M-Modes Using Heuristic Search , 2016, IJCAI.

[16]  Gregory Shakhnarovich,et al.  Diverse M-Best Solutions in Markov Random Fields , 2012, ECCV.

[17]  D. Nilsson,et al.  An efficient algorithm for finding the M most probable configurationsin probabilistic expert systems , 1998, Stat. Comput..

[18]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[19]  Menachem Fromer,et al.  Accurate prediction for atomic‐level protein design and its application in diversifying the near‐optimal sequence space , 2009, Proteins.

[20]  Stefanie Jegelka,et al.  Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets , 2014, NIPS.

[21]  S R Leeder,et al.  Changing prevalence of asthma in Australian children , 1994, BMJ.

[22]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.