Getting Feasible Variable Estimates from Infeasible Ones: MRF Local Polytope Study

This paper proposes a method for the construction of approximate feasible primal solutions from infeasible ones for large-scale optimization problems possessing certain separability properties. Whereas the infeasible primal estimates can typically be produced from (sub-) gradients of the dual function, it is often not easy to project them to the primal feasible set, since the projection itself has a complexity comparable to the complexity of the initial problem. We propose an alternative efficient method to obtain feasibility and show that its properties influencing the convergence to the optimum are similar to the properties of the Euclidean projection. We apply our method to the local polytope relaxation of inference problems for Markov Random Fields and discuss its advantages over existing methods.

[1]  Nikos Komodakis,et al.  MRF Optimization via Dual Decomposition: Message-Passing Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Christoph Schnörr,et al.  A bundle approach to efficient MAP-inference by Lagrangian relaxation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[4]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[5]  Stephen Gould,et al.  Accelerated dual decomposition for MAP inference , 2010, ICML.

[6]  Christoph Schnörr,et al.  A study of Nesterov's scheme for Lagrangian decomposition and MAP labeling , 2011, CVPR 2011.

[7]  Jeremy Jancsary,et al.  Convergent Decomposition Solvers for Tree-reweighted Free Energies , 2011, AISTATS.

[8]  Tommi S. Jaakkola,et al.  Introduction to dual composition for inference , 2011 .

[9]  Martin J. Wainwright,et al.  Message-passing for Graph-structured Linear Programs: Proximal Methods and Rounding Schemes , 2010, J. Mach. Learn. Res..

[10]  D. Sontag.  1 Introduction to Dual Decomposition for Inference , 2010 .

[11]  Christoph Schnörr,et al.  Efficient MRF Energy Minimization via Adaptive Diminishing Smoothing , 2012, UAI.

[12]  Dmitry M. Malioutov,et al.  Lagrangian Relaxation for MAP Estimation in Graphical Models , 2007, ArXiv.

[13]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[14]  Tommi S. Jaakkola,et al.  Tightening LP Relaxations for MAP using Message Passing , 2008, UAI.

[15]  Tomas Werner,et al.  Revisiting the Decomposition Approach to Inference in Exponential Families and Graphical Models , 2009 .

[16]  Martin J. Wainwright,et al.  A new class of upper bounds on the log partition function , 2002, IEEE Transactions on Information Theory.

[17]  Tamir Hazan,et al.  Tightening Fractional Covering Upper Bounds on the Partition Function for High-Order Region Graphs , 2012, UAI.

[18]  Ofer Meshi,et al.  An Alternating Direction Method for Dual MAP LP Relaxation , 2011, ECML/PKDD.

[19]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[20]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .

[22]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[23]  Yair Weiss,et al.  MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies , 2007, UAI.

[24]  Jörg H. Kappes,et al.  OpenGM: A C++ Library for Discrete Graphical Models , 2012, ArXiv.

[25]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26]  Tom Heskes,et al.  On the Uniqueness of Loopy Belief Propagation Fixed Points , 2004, Neural Computation.

[27]  Michael Patriksson,et al.  Ergodic, primal convergence in dual subgradient schemes for convex programming , 1999, Mathematical programming.

[28]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[29]  M. Shlezinger.  Syntactic analysis of two-dimensional visual signals in the presence of noise , 1976 .

[30]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[31]  Michail I. Schlesinger,et al.  Stop Condition for Subgradient Minimization in Dual Relaxed (max, +) Problem , 2011, EMMCVPR.

[32]  Hanif D. Sherali,et al.  Linear Programming and Network Flows , 1977 .

[33]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[35]  Tamir Hazan,et al.  Norm-Product Belief Propagation: Primal-Dual Message-Passing for Approximate Inference , 2009, IEEE Transactions on Information Theory.

[36]  Tommi S. Jaakkola,et al.  Convergence Rate Analysis of MAP Coordinate Minimization Algorithms , 2012, NIPS.

[37]  Eric P. Xing,et al.  An Augmented Lagrangian Approach to Constrained MAP Inference , 2011, ICML.

[38]  Nikos Komodakis,et al.  MRF Energy Minimization and Beyond via Dual Decomposition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Christoph Schnörr,et al.  Evaluation of a First-Order Primal-Dual Algorithm for MRF Energy Minimization , 2011, EMMCVPR.

[40]  Qiang Fu,et al.  Bethe-ADMM for Tree Decomposition based Parallel MAP Inference , 2013, UAI.

[41]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.