Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes

Consider communication over a binary-input memoryless output-symmetric channel with low-density parity-check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic formula for the average input-output conditional entropy per bit in the infinite block length limit. Montanari proved a lower bound for this entropy, in the case of LDPC ensembles with convex check degree polynomial, which matches the replica formula. Here we extend this lower bound to any irregular LDPC ensemble. The new feature of our work is an analysis of the second derivative of the conditional input-output entropy with respect to noise. A close relation arises between this second derivative and correlation or mutual information of codebits. This allows us to extend the realm of the ldquointerpolation method,rdquo in particular, we show how channel symmetry allows to control the fluctuations of the ldquooverlap parametersrdquo.

[1]  西森 秀稔.  Statistical physics of spin glasses and information processing : an introduction , 2001 .

[2]  Nicolas Macris,et al.  Sharp Bounds on Generalized EXIT Functions , 2007, IEEE Transactions on Information Theory.

[3]  F. Guerra.  Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model , 2002, cond-mat/0205123.

[4]  M. Talagrand.  The parisi formula , 2006 .

[5]  Igal Sason,et al.  On Achievable Rates and Complexity of LDPC Codes Over Parallel Channels: Bounds and Applications , 2007, IEEE Transactions on Information Theory.

[6]  Igal Sason,et al.  Parity-Check Density Versus Performance of Binary Linear Block Codes: New Bounds and Applications , 2007, IEEE Transactions on Information Theory.

[7]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[8]  S. Kudekar,et al.  Decay of correlations: An application to low-density parity check codes , 2008, 2008 5th International Symposium on Turbo Codes and Related Topics.

[9]  Michele Leone,et al.  Replica Bounds for Optimization Problems and Diluted Spin Systems , 2002 .

[10]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[11]  Nicolas Macris,et al.  Correlation inequalities: a useful tool in the theory of LDPC codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[12]  Saad,et al.  Typical performance of gallager-type error-correcting codes , 2000, Physical review letters.

[13]  M. Talagrand,et al.  Bounds for diluted mean-fields spin glass models , 2004, math/0405357.

[14]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[15]  Rüdiger L. Urbanke,et al.  Parity-check density versus performance of binary linear block codes over memoryless symmetric channels , 2003, IEEE Transactions on Information Theory.

[16]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[17]  Nicolas Macris,et al.  Proof of replica formulas in the high noise regime for communication using LDGM codes , 2008, 2008 IEEE Information Theory Workshop.

[18]  Pisa,et al.  Quadratic replica coupling in the Sherrington-Kirkpatrick mean field spin glass model , 2002, cond-mat/0201091.

[19]  Andrea Montanari,et al.  Maxwell's construction: the hidden bridge between maximum-likelihood and iterative decoding , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[20]  Andrea Montanari,et al.  Asymptotic Rate versus Design Rate , 2007, 2007 IEEE International Symposium on Information Theory.

[21]  Nicolas Macris,et al.  Sharp Bounds for MAP Decoding of General Irregular LDPC Codes , 2006, 2006 IEEE International Symposium on Information Theory.

[22]  Nicolas Macris,et al.  Exact solution of a p-spin model and its relationship to error correcting codes , 2006, 2006 IEEE International Symposium on Information Theory.

[23]  D. Burshtein,et al.  Upper bounds on the rate of LDPC codes , 2002, Proceedings IEEE International Symposium on Information Theory,.

[24]  Nicolas Macris,et al.  Exact solution for the conditional entropy of Poissonian LDPC codes over the Binary Erasure Channel , 2007, 2007 IEEE International Symposium on Information Theory.

[25]  Igal Sason,et al.  On Universal Properties of Capacity-Approaching LDPC Code Ensembles , 2007, IEEE Transactions on Information Theory.

[26]  M. Talagrand,et al.  Spin Glasses: A Challenge for Mathematicians , 2003 .

[27]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[28]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[29]  A. Montanari.  The glassy phase of Gallager codes , 2001, cond-mat/0104079.

[30]  Nicolas Macris,et al.  Decay of Correlations for Sparse Graph Error Correcting Codes , 2011, SIAM J. Discret. Math..

[31]  Nicolas Macris,et al.  Griffith–Kelly–Sherman Correlation Inequalities: A Useful Tool in the Theory of Error Correcting Codes , 2007, IEEE Transactions on Information Theory.