Probabilistic Analysis of Linear Programming Decoding

We initiate the probabilistic analysis of linear programming (LP) decoding of low-density parity-check (LDPC) codes. Specifically, we show that for a random LDPC code ensemble, the linear programming decoder of Feldman succeeds in correcting a constant fraction of errors with high probability. The fraction of correctable errors guaranteed by our analysis surpasses previous nonasymptotic results for LDPC codes, and in particular, exceeds the best previous finite-length result on LP decoding by a factor greater than ten. This improvement stems in part from our analysis of probabilistic bit-flipping channels, as opposed to adversarial channels. At the core of our analysis is a novel combinatorial characterization of LP decoding success, based on the notion of a flow on the Tanner graph of the code. An interesting by-product of our analysis is to establish the existence of ldquoprobabilistic expansionrdquo in random bipartite graphs, in which one requires only that almost every (as opposed to every) set of a certain size expands, for sets much larger than in the classical worst case setting.

[1]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Richard M. Wilson,et al.  A course in combinatorics , 1992 .

[6]  N. Fisher,et al.  Probability Inequalities for Sums of Bounded Random Variables , 1994 .

[7]  Daniel A. Spielman,et al.  Expander codes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[8]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[9]  Sae-Young Chung,et al.  Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation , 2001, IEEE Trans. Inf. Theory.

[10]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[11]  David Burshtein,et al.  Expander graph arguments for message-passing algorithms , 2001, IEEE Trans. Inf. Theory.

[12]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[13]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[14]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[15]  Daniel Bienstock,et al.  Potential Function Methods for Approximately Solving Linear Programming Problems: Theory and Practice , 2002 .

[16]  Martin J. Wainwright,et al.  Linear Programming-Based Decoding of Turbo-Like Codes and its Relation to Iterative Approaches , 2002 .

[17]  Jon Feldman,et al.  Decoding turbo-like codes via linear programming , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[18]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[19]  P. Vontobel,et al.  Graph-covers and iterative decoding of nite length codes , 2003 .

[20]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[21]  A. Barg,et al.  Error Exponents of Expander Codes under Linear-Complexity Decoding , 2004, SIAM J. Discret. Math..

[22]  Jon Feldman,et al.  LP decoding achieves capacity , 2005, SODA '05.

[23]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[24]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[25]  Guy Even,et al.  Improved bounds on the word error probability of RA(2) codes with linear-programming-based decoding , 2005, IEEE Transactions on Information Theory.

[26]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[27]  Alexandros G. Dimakis,et al.  Guessing Facets: Polytope Structure and Improved LP Decoding , 2006, ISIT.

[28]  Ralf Koetter,et al.  Bounds on the Threshold of Linear Programming Decoding , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[29]  Michael Chertkov,et al.  An Efficient Pseudo-Codeword Search Algorithm for Linear Programming Decoding of LDPC Codes , 2006, ArXiv.

[30]  Ralf Koetter,et al.  Towards Low-Complexity Linear-Programming Decoding , 2006, ArXiv.

[31]  Paul H. Siegel,et al.  Adaptive Linear Programming Decoding , 2006, 2006 IEEE International Symposium on Information Theory.

[32]  Stark C. Draper,et al.  ML decoding via mixed-integer adaptive linear programming , 2007, 2007 IEEE International Symposium on Information Theory.

[33]  Michael Chertkov,et al.  Reducing the Error Floor , 2007, 2007 IEEE Information Theory Workshop.

[34]  Jon Feldman,et al.  Cascaded Formulation of the Fundamental Polytope of General Linear Block Codes , 2007, 2007 IEEE International Symposium on Information Theory.

[35]  Deepak Sridhara,et al.  Eigenvalue bounds on the pseudocodeword weight of expander codes , 2007, Adv. Math. Commun..

[36]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[37]  Michael Chertkov,et al.  An Efficient Pseudocodeword Search Algorithm for Linear Programming Decoding of LDPC Codes , 2006, IEEE Transactions on Information Theory.