Lazy Generic Cuts

An efficient algorithm for inference in binary higher order MRF-MAP is proposed.Scalable to mid sized cliques which could not be done by state of the art.The algorithm is a flow based combinatorial algorithm based on Generic Cuts.In a typical inference problem minimum depends only on small set of constraints.The experiments validate the observation and show superiority over state of the art. LP relaxation based message passing and flow-based algorithms are two of the popular techniques for performing MAP inference in graphical models. Generic Cuts (GC) (Arora et?al., 2015) combines the two approaches to generalize the traditional max-flow min-cut based algorithms for binary models with higher order clique potentials. The algorithm has been shown to be significantly faster than the state of the art algorithms. The time and memory complexities of Generic Cuts are linear in the number of constraints, which in turn is exponential in the clique size. This limits the applicability of the approach to small cliques only. In this paper, we propose a lazy version of Generic Cuts exploiting the property that in most of such inference problems a large fraction of the constraints are never used during the course of minimization. We start with a small set of constraints (called the active constraints) which are expected to play a role during the minimization process. GC is then run with this reduced set allowing it to be efficient in time and memory. The set of active constraints is adaptively learnt over multiple iterations while guaranteeing convergence to the optimum for submodular clique potentials. Our experiments show that the number of constraints required by the algorithm is typically less than 3% of the total number of constraints. Experiments on computer vision datasets show that our approach can significantly outperform the state of the art both in terms of time and memory and is scalable to clique sizes that could not be handled by existing approaches.

[1]  Endre Boros,et al.  A graph cut algorithm for higher-order Markov Random Fields , 2011, 2011 International Conference on Computer Vision.

[2]  S. N. Maheshwari,et al.  Multi-label Generic Cuts: Optimal Inference in Multi-label Multi-clique MRF-MAP Problems , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Nikos Komodakis,et al.  MRF Optimization via Dual Decomposition: Message-Passing Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[4]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[5]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Nikos Komodakis,et al.  Beyond pairwise energies: Efficient optimization for higher-order MRFs , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Subhashis Banerjee,et al.  An Efficient Graph Cut Algorithm for Computer Vision Problems , 2010, ECCV.

[8]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Pushmeet Kohli,et al.  Dynamic Graph Cuts for Efficient Inference in Markov Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Satoru Iwata,et al.  A simple combinatorial algorithm for submodular function minimization , 2009, SODA.

[11]  Richard S. Zemel,et al.  HOP-MAP: Efficient Message Passing with High Order Potentials , 2010, AISTATS.

[12]  Sebastian Riedel Improving the Accuracy and Efficiency of MAP Inference for Markov Logic , 2008, UAI.

[13]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[14]  Vladimir Kolmogorov,et al.  Minimizing Nonsubmodular Functions with Graph Cuts-A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Nikos Komodakis,et al.  Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey , 2013, Comput. Vis. Image Underst..

[16]  Thorsten Joachims,et al.  Structured Learning of Sum-of-Submodular Higher Order Energy Functions , 2013, 2013 IEEE International Conference on Computer Vision.

[17]  Subhashis Banerjee,et al.  Generalized Flows for Optimal Inference in Higher Order MRF-MAP , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Pushmeet Kohli,et al.  Minimizing sparse higher order energy functions of discrete variables , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Hiroshi Ishikawa,et al.  Higher-Order Clique Reduction without Auxiliary Variables , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Chen Wang,et al.  A Primal-Dual Algorithm for Higher-Order Multilabel Markov Random Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hiroshi Ishikawa,et al.  Transformation of General Binary MRF Minimization to the First-Order Case , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Andreas Krause,et al.  Efficient Minimization of Decomposable Submodular Functions , 2010, NIPS.

[25]  Satoru Iwata,et al.  A fully combinatorial algorithm for submodular function minimization , 2001, SODA '02.

[26]  Fredrik Kahl,et al.  Generalized roof duality , 2012, Discret. Appl. Math..

[27]  Vladimir Kolmogorov,et al.  Minimizing a sum of submodular functions , 2010, Discret. Appl. Math..

[28]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.

[29]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[30]  David Sontag,et al.  Efficiently Searching for Frustrated Cycles in MAP Inference , 2012, UAI.

[31]  Tommi S. Jaakkola,et al.  Introduction to dual composition for inference , 2011 .

[32]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..