Iterative decoding beyond belief propagation

At the heart of modern coding theory lies the fact that low-density parity-check (LDPC) codes can be efficiently decoded by belief propagation (BP). The BP is an inference algorithm which operates on a graphical model of a code, and lends itself to low-complexity and high-speed implementations, making it the algorithm of choice in many applications. It has unprecedentedly good error rate performance, so good that when decoded by the BP, LDPC codes approach theoretical limits of channel capacity. However, this capacity approaching property holds only in the asymptotic limit of code length, while codes of practical lengths suffer abrupt performance degradation in the low noise regime known as the error floor phenomenon. Our study of error floor has led to an interesting and surprising finding that it is possible to design iterative decoders which are much simpler yet better than belief propagation! These decoders do not propagate beliefs but a rather different kind of messages that reflect the local structure of the code graph. This has opened a plethora of exciting theoretical problems and applications. This paper introduces this new paradigm.

[1]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[2]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[3]  Thomas J. Richardson,et al.  Error Floors of LDPC Codes , 2003 .

[4]  Jeremy Thorpe,et al.  Memory-efficient decoding of LDPC codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[5]  F.R. Kschischang,et al.  Low-Density Parity-Check Codes for Discretized Min-Sum Decoding , 2006, 23rd Biennial Symposium on Communications, 2006.

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[7]  Shashi Kiran Chilappagari,et al.  Error Floors of LDPC Codes on the Binary Symmetric Channel , 2006, 2006 IEEE International Conference on Communications.

[8]  B. Vasic,et al.  Trapping set ontology , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  Johannes B. Huber,et al.  The Trapping Redundancy of Linear Block Codes , 2006, IEEE Transactions on Information Theory.

[10]  Shashi Kiran Chilappagari,et al.  Eliminating trapping sets in low-density parity-check codes by using Tanner graph covers , 2008, IEEE Transactions on Information Theory.

[11]  Dror Weitz,et al.  Counting independent sets up to the tree threshold , 2006, STOC '06.

[12]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[13]  Andrea Montanari,et al.  TP Decoding , 2007, ArXiv.

[14]  Niclas Wiberg,et al.  Codes and Decoding on General Graphs , 1996 .

[15]  Johannes B. Huber,et al.  Multiple-Bases Belief-Propagation for Decoding of Short Block Codes , 2007, 2007 IEEE International Symposium on Information Theory.

[16]  Ajay Dholakia,et al.  Reduced-complexity decoding of LDPC codes , 2005, IEEE Transactions on Communications.

[17]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[18]  Aleksandar Kavcic,et al.  Augmented Belief Propagation Decoding of Low-Density Parity Check Codes , 2006, IEEE Transactions on Communications.

[19]  Hideki Imai,et al.  Reduced complexity iterative decoding of low-density parity check codes based on belief propagation , 1999, IEEE Trans. Commun..

[20]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[21]  Michael Chertkov,et al.  Loop Calculus Helps to Improve Belief Propagation and Linear Programming Decodings of Low-Density-Parity-Check Codes , 2006, ArXiv.

[22]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[23]  Stark C. Draper,et al.  Multi-stage decoding of LDPC codes , 2009, 2009 IEEE International Symposium on Information Theory.

[24]  Brendan J. Frey,et al.  Signal-space characterization of iterative decoding , 2001, IEEE Trans. Inf. Theory.

[25]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[26]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[27]  David Declercq,et al.  Multilevel decoders surpassing belief propagation on the binary symmetric channel , 2010, 2010 IEEE International Symposium on Information Theory.

[28]  Shashi Kiran Chilappagari,et al.  LDPC codes which can correct three errors under iterative decoding , 2008, 2008 IEEE Information Theory Workshop.

[29]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.