Maximum Persistency via Iterative Relaxed Inference in Graphical Models

We consider the NP-hard problem of MAP-inference for undirected discrete graphical models. We propose a polynomial time and practically efficient algorithm for finding a part of its optimal solution. Specifically, our algorithm marks some labels of the considered graphical model either as (i) optimal, meaning that they belong to all optimal solutions of the inference problem; (ii) non-optimal if they provably do not belong to any solution. With access to an exact solver of a linear programming relaxation to the MAP-inference problem, our algorithm marks the maximal possible (in a specified sense) number of labels. We also present a version of the algorithm, which has access to a suboptimal dual solver only and still can ensure the (non-)optimality for the marked labels, although the overall number of the marked labels may decrease. We propose an efficient implementation, which runs in time comparable to a single run of a suboptimal dual solver. Our method is well-scalable and shows state-of-the-art results on computational benchmarks from machine learning and computer vision.

[1]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[3]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[4]  Vladimir Kolmogorov,et al.  Potts Model, Parametric Maxflow and K-Submodular Functions , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Christoph Schnörr,et al.  Partial Optimality via Iterative Pruning for the Potts Model , 2013, SSVM.

[8]  M. I. Schlesingera,et al.  Diffusion algorithms and structural recognition optimization problems , 2011 .

[9]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Alexander Shekhovtsov,et al.  Maximum Persistency in Energy Minimization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yair Weiss,et al.  Minimizing and Learning Energy Functions for Side-Chain Prediction , 2008, J. Comput. Biol..

[13]  Christoph Schnörr,et al.  Efficient MRF Energy Minimization via Adaptive Diminishing Smoothing , 2012, UAI.

[14]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems , 2014, International Journal of Computer Vision.

[15]  Christopher Zach,et al.  A Principled Approach for Coarse-to-Fine MAP Inference , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Christoph Schnörr,et al.  Partial Optimality by Pruning for MAP-Inference with General Graphical Models , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Endre Boros,et al.  Pseudo-Boolean optimization , 2002, Discret. Appl. Math..

[18]  Pushmeet Kohli,et al.  Reduce, reuse & recycle: Efficiently solving multi-label MRFs , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Vladimir Kolmogorov,et al.  Generalized roof duality and bisubmodular functions , 2010, Discret. Appl. Math..

[20]  Vladimir Kolmogorov,et al.  On partial optimality in multi-label MRFs , 2008, ICML '08.

[21]  Alexander Shekhovtsov,et al.  Higher order maximum persistency and comparison theorems , 2015, Comput. Vis. Image Underst..

[22]  WernerTomas.  A Linear Programming Approach to Max-Sum Problem , 2007 .

[23]  Johan Desmet,et al.  The dead-end elimination theorem and its use in protein side-chain positioning , 1992, Nature.

[24]  M. Shlezinger.  Syntactic analysis of two-dimensional visual signals in the presence of noise , 1976 .

[25]  Christoph Schnörr,et al.  Partial Optimality by Pruning for MAP-Inference with General Graphical Models , 2014, CVPR.

[26]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[28]  Hanif D. Sherali,et al.  Persistency in 0-1 Polynomial Programming , 1998, Math. Oper. Res..

[29]  M. I. Schlesinger,et al.  Some solvable subclasses of structural recognition problems , 2000 .

[30]  Christoph Schnörr,et al.  A study of Nesterov's scheme for Lagrangian decomposition and MAP labeling , 2011, CVPR 2011.

[31]  Bogdan Savchynskyy,et al.  Getting Feasible Variable Estimates from Infeasible Ones: MRF Local Polytope Study , 2012, 2013 IEEE International Conference on Computer Vision Workshops.

[32]  A. Shekhovtsov.  Exact and Partial Energy Minimization in Computer Vision , 2013 .

[33]  Daniel Cremers,et al.  Generalized Roof Duality for Multi-Label Optimization: Optimal Lower Bounds and Persistency , 2012, ECCV.