Using linear programming to Decode Binary linear codes

A new method is given for performing approximate maximum-likelihood (ML) decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel. The decoding algorithm is based on a linear programming (LP) relaxation that is defined by a factor graph or parity-check representation of the code. The resulting "LP decoder" generalizes our previous work on turbo-like codes. A precise combinatorial characterization of when the LP decoder succeeds is provided, based on pseudocodewords associated with the factor graph. Our definition of a pseudocodeword unifies other such notions known for iterative algorithms, including "stopping sets," "irreducible closed walks," "trellis cycles," "deviation sets," and "graph covers." The fractional distance d/sub frac/ of a code is introduced, which is a lower bound on the classical distance. It is shown that the efficient LP decoder will correct up to /spl lceil/d/sub frac//2/spl rceil/-1 errors and that there are codes with d/sub frac/=/spl Omega/(n/sup 1-/spl epsi//). An efficient algorithm to compute the fractional distance is presented. Experimental evidence shows a similar performance on low-density parity-check (LDPC) codes between LP decoding and the min-sum and sum-product algorithms. Methods for tightening the LP relaxation to improve performance are also provided.

[1]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[2]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[3]  D.J.C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[4]  G. Forney,et al.  Iterative Decoding of Tail-Biting Trellises and Connections with Symbolic Dynamics , 2001 .

[5]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[6]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[7]  P. Vontobel,et al.  Constructions of LDPC Codes using Ramanujan Graphs and Ideas from Margulis , 2000 .

[8]  P. Vontobel,et al.  Graph-covers and iterative decoding of nite length codes , 2003 .

[9]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[10]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[11]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[12]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[13]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[14]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[15]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[16]  R. Koetter,et al.  On the Effective Weights of Pseudocodewords for Codes Defined on Graphs with Cycles , 2001 .

[17]  Jon Feldman,et al.  Decoding turbo-like codes via linear programming , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[18]  Alexander Schrijver,et al.  Cones of Matrices and Set-Functions and 0-1 Optimization , 1991, SIAM J. Optim..

[19]  Robert G. Jeroslow.  On defining sets of vertices of the hypercube by linear inequalities , 1975, Discret. Math..

[20]  Jon Feldman,et al.  LP decoding achieves capacity , 2005, SODA '05.

[21]  Hanif D. Sherali,et al.  A Hierarchy of Relaxations Between the Continuous and Convex Hull Representations for Zero-One Programming Problems , 1990, SIAM J. Discret. Math..

[22]  Warren P. Adams,et al.  A hierarchy of relaxation between the continuous and convex hull representations , 1990 .

[23]  Ralf Koetter,et al.  Lower bounds on the minimum pseudoweight of linear codes , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[24]  Niclas Wiberg,et al.  Codes and Decoding on General Graphs , 1996 .

[25]  Martin J. Wainwright,et al.  Linear Programming-Based Decoding of Turbo-Like Codes and its Relation to Iterative Approaches , 2002 .

[26]  M. Wainwright,et al.  Using Linear Programming to Decode Linear Codes , 2003 .

[27]  Sae-Young Chung,et al.  On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit , 2001, IEEE Communications Letters.

[28]  David K. Smith.  Theory of Linear and Integer Programming , 1987 .

[29]  D. Spielman,et al.  Expander codes , 1996 .

[30]  M. Yannakakis.  Expressing combinatorial optimization problems by linear programs , 1991, Symposium on the Theory of Computing.

[31]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[32]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[33]  Emre Telatar,et al.  Finite-length analysis of low-density parity-check codes on the binary erasure channel , 2002, IEEE Trans. Inf. Theory.

[34]  Brendan J. Frey,et al.  Signal-space characterization of iterative decoding , 2001, IEEE Trans. Inf. Theory.

[35]  Jon Feldman,et al.  Decoding error-correcting codes via linear programming , 2003 .

[36]  M. Luby,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).