On Decoding Irregular Tanner Codes With Local-Optimality Guarantees

We consider decoding of binary linear Tanner codes using message-passing iterative decoding and linear-programming (LP) decoding in memoryless binary-input output-symmetric (MBIOS) channels. We present new certificates that are based on a combinatorial characterization for the local optimality of a codeword in irregular Tanner codes with respect to any MBIOS channel. This characterization is a generalization of (Arora , Proc. ACM Symp. Theory of Computing, 2009) and (Vontobel, Proc. Inf. Theory and Appl. Workshop, 2010) and is based on a conical combination of normalized weighted subtrees in the computation trees of the Tanner graph. These subtrees may have any finite height h (even equal or greater than half of the girth of the Tanner graph). In addition, the degrees of local-code nodes in these subtrees are not restricted to two (i.e., these subtrees are not restricted to skinny trees). We prove that local optimality in this new characterization implies maximum-likelihood (ML) optimality and LP optimality, and show that a certificate can be computed efficiently. We also present a new message-passing iterative decoding algorithm, called normalized weighted min-sum (NWMS). NWMS decoding is a belief-propagation (BP) type algorithm that applies to any irregular binary Tanner code with single parity-check local codes (e.g., low-density and high-density parity-check codes). We prove that if a locally optimal codeword with respect to height parameter h exists (whereby notably h is not limited by the girth of the Tanner graph), then NWMS decoding finds this codeword in h iterations. The decoding guarantee of the NWMS decoding algorithm applies whenever there exists a locally optimal codeword. Because local optimality of a codeword implies that it is the unique ML codeword, the decoding guarantee also provides an ML certificate for this codeword. Finally, we apply the new local-optimality characterization to regular Tanner codes, and prove lower bounds on the noise thresholds of LP decoding in MBIOS channels. When the noise is below these lower bounds, the probability that LP decoding fails to decode the transmitted codeword decays doubly exponentially in the girth of the Tanner graph.

[1]  Henry D. Pfister,et al.  Convergence of weighted min-sum decoding via dynamic programming on coupled trees , 2010, 2010 6th International Symposium on Turbo Codes & Iterative Information Processing.

[2]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[3]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[4]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[5]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[6]  Alexander Barg,et al.  Error exponents of expander codes , 2002, IEEE Trans. Inf. Theory.

[7]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[8]  Niclas Wiberg,et al.  Codes and Decoding on General Graphs , 1996 .

[9]  Jon Feldman,et al.  Decoding turbo-like codes via linear programming , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[10]  D.J.C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[12]  Ralf Koetter,et al.  On the Block Error Probability of LP Decoding of LDPC Codes , 2006, ArXiv.

[13]  Alexandros G. Dimakis,et al.  Probabilistic Analysis of Linear Programming Decoding , 2007, IEEE Transactions on Information Theory.

[14]  Pascal O. Vontobel,et al.  Counting in Graph Covers: A Combinatorial Characterization of the Bethe Entropy Function , 2010, IEEE Transactions on Information Theory.

[15]  Guy Even,et al.  Improved bounds on the word error probability of RA(2) codes with linear-programming-based decoding , 2005, IEEE Transactions on Information Theory.

[16]  Ron M. Roth,et al.  Generalized minimum distance iterative decoding of expander codes , 2003, Proceedings 2003 IEEE Information Theory Workshop (Cat. No.03EX674).

[17]  Alexander Barg,et al.  Error Exponents of Expander Codes under Linear-Complexity Decoding , 2004, SIAM J. Discret. Math..

[18]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[19]  Hans-Andrea Loeliger,et al.  Codes and iterative decoding on general graphs , 1995, Eur. Trans. Telecommun..

[20]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[21]  Guy Even,et al.  LP Decoding of Regular LDPC Codes in Memoryless Channels , 2010, IEEE Transactions on Information Theory.

[22]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[23]  Vitaly Skachek,et al.  Correcting a Fraction of Errors in Nonbinary Expander Codes With Linear Programming , 2009, IEEE Transactions on Information Theory.

[24]  P. Vontobel,et al.  Graph-Cover Decoding and Finite-Length Analysis of Message-Passing Iterative Decoding of LDPC Codes , 2005, ArXiv.

[25]  P. Vontobel,et al.  Constructions of LDPC Codes using Ramanujan Graphs and Ideas from Margulis , 2000 .

[26]  D. Spielman,et al.  Expander codes , 1996 .

[27]  Jon Feldman,et al.  LP decoding achieves capacity , 2005, SODA '05.

[28]  Pascal O. Vontobel,et al.  A factor-graph-based random walk, and its relevance for LP decoding analysis and Bethe entropy characterization , 2010, 2010 Information Theory and Applications Workshop (ITA).

[29]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[30]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[31]  Henry D. Pfister,et al.  Convergence of Weighted Min-Sum Decoding Via Dynamic Programming on Trees , 2011, IEEE Transactions on Information Theory.

[32]  Ajay Dholakia,et al.  Reduced-complexity decoding of LDPC codes , 2005, IEEE Transactions on Communications.

[33]  Jon Feldman,et al.  Decoding error-correcting codes via linear programming , 2003 .

[34]  Jinghu Chen,et al.  Density evolution for two improved BP-Based decoding algorithms of LDPC codes , 2002, IEEE Communications Letters.

[35]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[36]  Sanjeev Arora,et al.  Message-Passing Algorithms and Improved LP Decoding , 2009, IEEE Transactions on Information Theory.

[37]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.