Guessing Facets: Polytope Structure and Improved LP Decoder

We investigate the structure of the polytope underlying the linear programming (LP) decoder introduced by Feldman, Karger, and Wainwright. We first show that for expander codes, every fractional pseudocodeword always has at least a constant fraction of nonintegral bits. We then prove that for expander codes, the active set of any fractional pseudocodeword is smaller by a constant fraction than that of any codeword. We further exploit these geometrical properties to devise an improved decoding algorithm with the same order of complexity as LP decoding that provably performs better. The method is very simple: it first applies ordinary LP decoding, and when it fails, it proceeds by guessing facets of the polytope, and then resolving the linear program on these facets. While the LP decoder succeeds only if the ML codeword has the highest likelihood over all pseudocodewords, we prove that the proposed algorithm, when applied to suitable expander codes, succeeds unless there exists a certain number of pseudocodewords, all adjacent to the ML codeword on the LP decoding polytope, and with higher likelihood than the ML codeword. We then describe an extended algorithm, still with polynomial complexity, that succeeds as long as there are at most polynomially many pseudocodewords above the ML codeword.

[1]  Ralf Koetter,et al.  Lower bounds on the minimum pseudoweight of linear codes , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[2]  Daniel A. Spielman,et al.  Expander codes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[3]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[4]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[5]  Michael Chertkov,et al.  Reducing the Error Floor , 2007, 2007 IEEE Information Theory Workshop.

[6]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[7]  Stark C. Draper,et al.  ML decoding via mixed-integer adaptive linear programming , 2007, 2007 IEEE International Symposium on Information Theory.

[8]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[9]  Jon Feldman,et al.  LP decoding achieves capacity , 2005, SODA '05.

[10]  Ralf Koetter,et al.  Towards Low-Complexity Linear-Programming Decoding , 2006, ArXiv.

[11]  Paul H. Siegel,et al.  Adaptive Linear Programming Decoding , 2006, 2006 IEEE International Symposium on Information Theory.

[12]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[13]  Alexandros G. Dimakis,et al.  Probabilistic Analysis of Linear Programming Decoding , 2007, IEEE Transactions on Information Theory.

[14]  Marc P. C. Fossorier,et al.  Iterative reliability-based decoding of low-density parity check codes , 2001, IEEE J. Sel. Areas Commun..

[15]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[16]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[17]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[18]  Jon Feldman,et al.  The benefit of thresholding in LP decoding of LDPC codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[19]  Jon Feldman,et al.  Cascaded Formulation of the Fundamental Polytope of General Linear Block Codes , 2007, 2007 IEEE International Symposium on Information Theory.

[20]  Michael I. Jordan Graphical Models , 2003 .

[21]  G. Ziegler Lectures on Polytopes , 1994 .

[22]  Andrea Montanari,et al.  Solving Constraint Satisfaction Problems through Belief Propagation-guided decimation , 2007, ArXiv.

[23]  Martin J. Wainwright,et al.  Linear Programming-Based Decoding of Turbo-Like Codes and its Relation to Iterative Approaches , 2002 .

[24]  Martin J. Wainwright,et al.  Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[25]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[26]  Faramarz Fekri,et al.  On decoding of low-density parity-check codes over the binary erasure channel , 2004, IEEE Transactions on Information Theory.

[27]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[28]  Jon Feldman,et al.  Nonlinear programming approaches to decoding low-density parity-check codes , 2006, IEEE Journal on Selected Areas in Communications.

[29]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[30]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[31]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[32]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[33]  Alexander Barg,et al.  Error exponents of expander codes , 2002, IEEE Trans. Inf. Theory.

[34]  P. Vontobel,et al.  Graph-covers and iterative decoding of nite length codes , 2003 .

[35]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[36]  Stephan ten Brink,et al.  Extrinsic information transfer functions: model and erasure channel properties , 2004, IEEE Transactions on Information Theory.