Gallager error-correcting codes for binary asymmetric channels

We derive critical noise levels for Gallager codes on asymmetric channels as a function of the input bias and the temperature. Using a statistical mechanics approach we study the space of codewords and the entropy in the various decoding regimes. We further discuss the relation of the convergence of the message passing algorithm with the endogenous property and complexity, characterizing solutions of recursive equations of distributions for cavity fields.

[1]  D. Aldous,et al.  A survey of max-type recursive distributional equations , 2004, math/0401388.

[2]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[3]  Yoshiyuki Kabashima,et al.  Survey propagation for the cascading Sourlas code , 2006, ArXiv.

[4]  R. Monasson.  Optimization problems and replica symmetry breaking in finite connectivity spin glasses , 1997, cond-mat/9707089.

[5]  D. Saad,et al.  Magnetization enumerator of real-valued symmetric channels in Gallager error-correcting codes. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  S. Franz,et al.  Dynamic phase transition for decoding algorithms. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  David J. C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[8]  D. Saad,et al.  Statistical mechanics of low-density parity-check codes , 2004 .

[9]  David Saad,et al.  Finite-connectivity spin-glass phase diagrams and low-density parity check codes. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  D. Saad,et al.  Statistical mechanics of error-correcting codes , 1999 .

[11]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[12]  David Burshtein,et al.  On the application of LDPC codes to arbitrary discrete-memoryless channels , 2003, IEEE Transactions on Information Theory.

[13]  Jean-Philippe Bouchaudyx.  Universality classes for extreme-value statistics , 1997 .

[14]  H. Nishimori.  Optimum Decoding Temperature for Error-Correcting Codes , 1993 .

[15]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .

[16]  Kanter,et al.  Mean-field theory of spin-glasses with finite coordination number. , 1987, Physical review letters.

[17]  I. Reed,et al.  Polynomial Codes Over Certain Finite Fields , 1960 .

[18]  M. Mézard,et al.  Universality classes for extreme value statistics , 1997, cond-mat/9707047.

[19]  Gerhard Lakemeyer,et al.  Exploring artificial intelligence in the new millennium , 2003 .

[20]  E. Gardner.  Spin glasses with p-spin interactions , 1985 .

[21]  J. van Mourik,et al.  Average and reliability error exponents in low-density parity-check codes , 2003 .

[22]  H. Vincent Poor,et al.  Density evolution for asymmetric memoryless channels , 2005, IEEE Transactions on Information Theory.

[23]  Olivier Rivoire.  Phases vitreuses, optimisation et grandes déviations , 2005 .

[24]  Andrea Montanari,et al.  Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View , 2007, ArXiv.

[25]  Rüdiger L. Urbanke,et al.  The renaissance of Gallager's low-density parity-check codes , 2003, IEEE Commun. Mag..

[26]  Cavity method for supersymmetry-breaking spin glasses , 2004, cond-mat/0407440.

[27]  D. Sherrington,et al.  Graph bipartitioning and spin glasses on a random network of fixed finite valence , 1987 .

[28]  B. Derrida.  Random-Energy Model: Limit of a Family of Disordered Models , 1980 .

[29]  M. Mézard,et al.  Frozen glass phase in the multi-index matching problem. , 2004, Physical review letters.

[30]  M. Mézard,et al.  Two Solutions to Diluted p-Spin Models and XORSAT Problems , 2003 .

[31]  A. Montanari.  The glassy phase of Gallager codes , 2001, cond-mat/0104079.

[32]  A. Montanari,et al.  On the nature of the low-temperature phase in discontinuous mean-field spin glasses , 2003, cond-mat/0301591.

[33]  M. Mézard,et al.  The Bethe lattice spin glass revisited , 2000, cond-mat/0009418.

[34]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[35]  A. Montanari.  Turbo codes: the phase transition , 2000, cond-mat/0003218.

[36]  Yoshiyuki Kabashima,et al.  Statistical physics of irregular low-density parity-check codes , 2000 .

[37]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[38]  Riccardo Zecchina,et al.  Survey propagation: An algorithm for satisfiability , 2002, Random Struct. Algorithms.

[39]  M. Mézard,et al.  A ferromagnet with a glass transition , 2001, cond-mat/0103026.

[40]  Thierry Mora,et al.  Statistical mechanics of error exponents for error-correcting codes , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Toshiyuki Tanaka,et al.  Typical performance of regular low-density parity-check codes over general symmetric channels , 2003 .

[42]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[43]  Saad,et al.  Statistical physics of regular low-density parity-check error-correcting codes , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[44]  L. Zdeborová,et al.  Thermodynamic origin of order parameters in mean‐field models of spin glasses , 2005, cond-mat/0504132.

[45]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[46]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[47]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[48]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[49]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[50]  Y. Iba.  The Nishimori line and Bayesian statistics , 1998, cond-mat/9809190.

[51]  Sae-Young Chung,et al.  On the capacity of low-density parity-check codes , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[52]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[53]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[54]  Andrea Montanari,et al.  The statistical mechanics of turbo codes , 1999 .

[55]  B Wemmenhove,et al.  Survey propagation at finite temperature: application to a Sourlas code as a toy model , 2006 .

[56]  Rüdiger L. Urbanke,et al.  Design of capacity-approaching irregular low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[57]  H. Nishimori.  Internal Energy, Specific Heat and Correlation Function of the Bond-Random Ising Model , 1981 .

[58]  A. Bray,et al.  Phase diagrams for dilute spin glasses , 1985 .

[59]  R Vicente,et al.  Finite-connectivity systems as error-correcting codes. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.