Facet Guessing for Finding the M-Best Integral Solutions of a Linear Program

In machine learning, it is often advantageous to return multiple solutions to an optimization problem, rather than the single optimal solution. We consider the M -best LP problem, where the goal is to find the M vertices of a polytope that minimize a linear cost function. We present an algorithm that can find the M -best when M is constant. We complement this by a hardness of approximation result. Furthermore, we study the problem of recovering the highest vertex with integral coordinates. This question arises naturally for MAP inference in graphical models. We show that for any polytope contained inside the unit hypercube, it is possible to find the highest integral vertex if it lies within the top poly(n) vertices of the LP. This allows us to avoid polynomially many fractional vertices and still recover the optimal vertex with {0, 1} coordinates. Our work resolves an open problem by Dimakis, Gohari, and Wainwright.

[1]  Thomas M. Liebling,et al.  Analysis of Backtrack Algorithms for Listing All Vertices and All Faces of a Convex Polyhedron , 1997, Comput. Geom..

[2]  Michael I. Jordan.  Graphical Models , 2003 .

[3]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[4]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[5]  Alexandros G. Dimakis,et al.  Guessing Facets: Polytope Structure and Improved LP Decoder , 2009, IEEE Transactions on Information Theory.

[6]  Alexandros G. Dimakis,et al.  Unequal Growth Codes: Intermediate Performance and Unequal Error Protection for Video Streaming , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[7]  P. Vontobel,et al.  Characterizations of pseudo-codewords of (low-density) parity-check codes , 2007 .

[8]  Alexandros G. Dimakis,et al.  Probabilistic Analysis of Linear Programming Decoding , 2007, IEEE Transactions on Information Theory.

[9]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[10]  Michael Chertkov,et al.  An Efficient Pseudocodeword Search Algorithm for Linear Programming Decoding of LDPC Codes , 2006, IEEE Transactions on Information Theory.

[11]  Amir Globerson,et al.  An LP View of the M-best MAP problem , 2009, NIPS.

[12]  David Burshtein.  Iterative approximate linear programming decoding of LDPC codes with linear complexity , 2009, IEEE Trans. Inf. Theory.

[13]  Dan Klein,et al.  Top-Down K-Best A* Parsing , 2010, ACL.

[14]  Deva Ramanan,et al.  N-best maximal decoders for part models , 2011, 2011 International Conference on Computer Vision.

[15]  Paul H. Siegel,et al.  Adaptive Cut Generation Algorithm for Improved Linear Programming Decoding of Binary Linear Codes , 2011, IEEE Transactions on Information Theory.

[16]  Stark C. Draper,et al.  Suppressing pseudocodewords by penalizing the objective of LP decoding , 2012, 2012 IEEE Information Theory Workshop.

[17]  Gregory Shakhnarovich,et al.  Diverse M-Best Solutions in Markov Random Fields , 2012, ECCV.

[18]  Pushmeet Kohli,et al.  Multiple Choice Learning: Learning to Produce Multiple Structured Outputs , 2012, NIPS.

[19]  Sanjeev Arora,et al.  Message-Passing Algorithms and Improved LP Decoding , 2012, IEEE Trans. Inf. Theory.

[20]  David Eppstein,et al.  K-Best Enumeration , 2014, Bull. EATCS.

[21]  Volker Kaibel,et al.  Forbidden Vertices , 2013, Math. Oper. Res..

[22]  Adrian Weller,et al.  Tightness of LP Relaxations for Almost Balanced Models , 2016, AISTATS.

[23]  Rina Dechter,et al.  Searching for the M Best Solutions in Graphical Models , 2016, J. Artif. Intell. Res..

[24]  Ofer Meshi,et al.  Train and Test Tightness of LP Relaxations in Structured Prediction , 2015, ICML.