Decay of Correlations for Sparse Graph Error Correcting Codes

The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely, low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code-bits and its relationship to the suboptimal belief propagation decoder are investigated. We consider the correlation (or covariance) between two code-bits, averaged over the noise realizations, as a function of the graph distance for the optimal decoder. Our main result is that this correlation decays exponentially fast for given low-density generator-matrix codes and a high enough noise parameter and also for given low-density parity-check codes and a low enough noise parameter. This has many consequences. Appropriate performance curves—called generalized extrinsic information transfer (GEXIT) functions—of the belief propagation and optimal decoders match in high/low noise regimes. This means that i...

[1]  G.D. Forney,et al.  Codes on graphs: Normal realizations , 2000, IEEE Trans. Inf. Theory.

[2]  Nicolas Macris,et al.  Sharp Bounds on Generalized EXIT Functions , 2007, IEEE Transactions on Information Theory.

[3]  Nicolas Macris,et al.  Griffith–Kelly–Sherman Correlation Inequalities: A Useful Tool in the Theory of Error Correcting Codes , 2007, IEEE Transactions on Information Theory.

[4]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[5]  Stephan ten Brink,et al.  Extrinsic information transfer functions: model and erasure channel properties , 2004, IEEE Transactions on Information Theory.

[6]  Nicolas Macris,et al.  Sharp Bounds for MAP Decoding of General Irregular LDPC Codes , 2006, 2006 IEEE International Symposium on Information Theory.

[7]  Michael Chertkov,et al.  Loop series for discrete statistical models on graphs , 2006, ArXiv.

[8]  Rüdiger L. Urbanke,et al.  Exchange of Limits: Why Iterative Decoding Works , 2008, IEEE Transactions on Information Theory.

[9]  Pisa,et al.  Quadratic replica coupling in the Sherrington-Kirkpatrick mean field spin glass model , 2002, cond-mat/0201091.

[10]  Sekhar Tatikonda,et al.  Loopy Belief Propogation and Gibbs Measures , 2002, UAI.

[11]  Abel Klein,et al.  Taming Griffiths' singularities: Infinite differentiability of quenched correlation functions , 1995 .

[12]  Michele Leone,et al.  Replica Bounds for Optimization Problems and Diluted Spin Systems , 2002 .

[13]  M. Talagrand,et al.  Spin Glasses: A Challenge for Mathematicians , 2003 .

[14]  Andrea Montanari,et al.  Asymptotic Rate versus Design Rate , 2007, 2007 IEEE International Symposium on Information Theory.

[15]  Nicolas Macris,et al.  Exact solution for the conditional entropy of Poissonian LDPC codes over the Binary Erasure Channel , 2007, 2007 IEEE International Symposium on Information Theory.

[16]  Andrea Montanari,et al.  The Generalized Area Theorem and Some of its Consequences , 2005, IEEE Transactions on Information Theory.

[17]  S. Kudekar,et al.  Decay of correlations: An application to low-density parity check codes , 2008, 2008 5th International Symposium on Turbo Codes and Related Topics.

[18]  Nicolas Macris,et al.  Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes , 2008, IEEE Transactions on Information Theory.

[19]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[20]  Nicolas Macris,et al.  Decay of correlations in low density parity check codes: Low noise regime , 2009, 2009 IEEE International Symposium on Information Theory.

[21]  B. Scoppola,et al.  Statistical Mechanics Approach to Coding Theory , 1999 .

[22]  Alberto Berretti.  Some properties of random Ising models , 1985 .

[23]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[24]  Saad,et al.  Statistical physics of regular low-density parity-check error-correcting codes , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[25]  Hans-Otto Georgii,et al.  Gibbs Measures and Phase Transitions , 1988 .

[26]  Nicolas Macris,et al.  Proof of replica formulas in the high noise regime for communication using LDGM codes , 2008, 2008 IEEE Information Theory Workshop.