Reed–Muller Codes Achieve Capacity on Erasure Channels

We introduce a new approach to proving that a sequence of deterministic linear codes achieves capacity on an erasure channel under maximum a posteriori decoding. Rather than relying on the precise structure of the codes, our method exploits code symmetry. In particular, the technique applies to any sequence of linear codes where the blocklengths are strictly increasing, the code rates converge, and the permutation group of each code is doubly transitive. In other words, we show that symmetry alone implies near-optimal performance. An important consequence of this result is that a sequence of Reed–Muller codes with increasing blocklength and converging rate achieves capacity. This possibility has been suggested previously in the literature but it has only been proven for cases where the limiting code rate is 0 or 1. Moreover, these results extend naturally to all affine-invariant codes and, thus, to extended primitive narrow-sense BCH codes. This also resolves, in the affirmative, the existence question for capacity-achieving sequences of binary cyclic codes. The primary tools used in the proof are the sharp threshold property for symmetric monotone Boolean functions and the area theorem for extrinsic information transfer functions.

[1]  Tadao Kasami,et al.  On the weight structure of Reed-Muller codes , 1970, IEEE Trans. Inf. Theory.

[2]  E. Arkan,et al.  A performance comparison of polar codes and Reed-Muller codes , 2008, IEEE Communications Letters.

[3]  Tadao Kasami,et al.  New generalizations of the Reed-Muller codes-I: Primitive codes , 1968, IEEE Trans. Inf. Theory.

[4]  G. Kalai,et al.  Every monotone graph property has a sharp threshold , 1996 .

[5]  Rüdiger L. Urbanke,et al.  Spatially coupled ensembles universally achieve capacity under belief propagation , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[6]  Erdal Arikan,et al.  Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[7]  Amin Coja-Oghlan,et al.  The asymptotic k-SAT threshold , 2014, STOC.

[8]  Ramprasad Saptharishi,et al.  Decoding high rate Reed-Muller codes from random errors in near linear time , 2015, ArXiv.

[9]  Assaf Naor,et al.  Rigorous location of phase transitions in hard optimization problems , 2005, Nature.

[10]  Philippe Delsarte,et al.  On cyclic codes that are invariant under the general linear group , 1970, IEEE Trans. Inf. Theory.

[11]  F. Didier,et al.  A New Upper Bound on the Block Error Probability After Decoding Over the Erasure Channel , 2006, IEEE Transactions on Information Theory.

[12]  Thierry P. Berger,et al.  The permutation group of affine-invariant extended cyclic codes , 1996, IEEE Trans. Inf. Theory.

[13]  Allan Sly,et al.  Proof of the Satisfiability Conjecture for Large k , 2014, STOC.

[14]  S. Brink Convergence of iterative decoding , 1999 .

[15]  Nicolas Macris,et al.  Threshold Saturation for Spatially Coupled LDPC and LDGM Codes on BMS Channels , 2013, IEEE Transactions on Information Theory.

[16]  A. Robert Calderbank,et al.  Reed-muller codes achieve capacity on the quantum erasure channel , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[17]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .

[18]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[19]  Irving S. Reed,et al.  A class of multiple-error-correcting codes and the decoding scheme , 1954, Trans. IRE Prof. Group Inf. Theory.

[20]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[21]  S. Safra,et al.  On the hardness of approximating minimum vertex cover , 2005 .

[22]  Amnon Ta-Shma,et al.  Extractors from Reed-Muller codes , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[23]  Jean-Pierre Tillich,et al.  Computing the Algebraic Immunity Efficiently , 2006, FSE.

[24]  Claude Carlet,et al.  Algebraic immunity for cryptographically significant Boolean functions: analysis and construction , 2006, IEEE Transactions on Information Theory.

[25]  Thierry P. Berger,et al.  The Automorphism Groups of BCH Codes and of Some Affine-Invariant Codes Over Extension Fields , 1999, Des. Codes Cryptogr..

[26]  Gilles Zémor,et al.  The Gaussian isoperimetric inequality and decoding error probabilities for the Gaussian channel , 2002, IEEE Transactions on Information Theory.

[27]  Ronitt Rubinfeld,et al.  Self-testing/correcting for polynomials and for approximate functions , 1991, STOC '91.

[28]  G. Kalai,et al.  Threshold Phenomena and Influence , 2005 .

[29]  Daniel A. Spielman,et al.  Linear-time encodable and decodable error-correcting codes , 1995, STOC '95.

[30]  Anne Canteaut,et al.  On cryptographic properties of the cosets of R(1, m) , 2001, IEEE Trans. Inf. Theory.

[31]  Ilya Dumer,et al.  Recursive decoding and its performance for low-rate Reed-Muller codes , 2004, IEEE Transactions on Information Theory.

[32]  Rüdiger L. Urbanke,et al.  Reed-Muller Codes Achieve Capacity on the Binary Erasure Channel under MAP Decoding , 2015, ArXiv.

[33]  Shu Lin,et al.  Polynomial codes , 1968, IEEE Trans. Inf. Theory.

[34]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .

[35]  L. Russo An approximate zero-one law , 1982 .

[36]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[37]  Nathan Linial,et al.  Collective coin flipping, robust voting schemes and minima of Banzhaf values , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[38]  N. Linial,et al.  The influence of variables in product spaces , 1992 .

[39]  Ilya Dumer,et al.  Soft-decision decoding of Reed-Muller codes: recursive lists , 2006, IEEE Transactions on Information Theory.

[40]  Shachar Lovett,et al.  Weight Distribution and List-Decoding Size of Reed–Muller Codes , 2012, IEEE Transactions on Information Theory.

[41]  Tadao Kasami,et al.  On the Weight Enumeration of Weights Less than 2.5d of Reed-Muller Codes , 1976, Inf. Control..

[42]  Madhu Sudan,et al.  Highly Resilient Correctors for Polynomials , 1992, Inf. Process. Lett..

[43]  Daniel J. Costello,et al.  Channel coding: The road to channel capacity , 2006, Proceedings of the IEEE.

[44]  N. J. A. Sloane,et al.  Weight enumerator for second-order Reed-Muller codes , 1970, IEEE Trans. Inf. Theory.

[45]  Ilya Dumer,et al.  Soft-decision decoding of Reed-Muller codes: a simplified algorithm , 2006, IEEE Transactions on Information Theory.

[46]  Amnon Ta-Shma,et al.  Extractors from Reed-Muller Codes , 2001, Electron. Colloquium Comput. Complex..

[47]  M. Darnell,et al.  Error Control Coding: Fundamentals and Applications , 1985 .

[48]  Uri Erez,et al.  Cyclic coded integer-forcing equalization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[49]  W. Cary Huffman,et al.  Fundamentals of Error-Correcting Codes , 1975 .

[50]  Yuan Zhou Introduction to Coding Theory , 2010 .

[51]  Ilya Dumer,et al.  Erasure Correction Performance of Linear Block Codes , 1993, Algebraic Coding.

[52]  Edward F. Assmus The Category of Linear Codes , 1998, IEEE Trans. Inf. Theory.

[53]  Michael Viderman,et al.  Locally Testable vs. Locally Decodable Codes , 2010, APPROX-RANDOM.

[54]  Pascal O. Vontobel,et al.  Algebraic coding for iterative decoding , 2003 .

[55]  Avi Wigderson,et al.  Reed-Muller Codes for Random Erasures and Errors , 2015, IEEE Trans. Inf. Theory.

[56]  Ramprasad Saptharishi,et al.  Efficiently Decoding Reed–Muller Codes From Random Errors , 2015, IEEE Transactions on Information Theory.

[57]  Claude Carlet,et al.  On Correlation-Immune Functions , 1991, CRYPTO.

[58]  Claude Carlet,et al.  On the construction of balanced boolean functions with a good algebraic immunity , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[59]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[60]  Gilles Zémor Threshold effects in codes , 1993, Algebraic Coding.

[61]  M. Talagrand Isoperimetry, logarithmic sobolev inequalities on the discrete cube, and margulis' graph connectivity theorem , 1993 .

[62]  Stephan ten Brink,et al.  Extrinsic information transfer functions: model and erasure channel properties , 2004, IEEE Transactions on Information Theory.

[63]  E. Wright,et al.  An Introduction to the Theory of Numbers , 1939 .

[64]  Michael Lentmaier,et al.  Iterative Decoding Threshold Analysis for LDPC Convolutional Codes , 2010, IEEE Transactions on Information Theory.

[65]  David E. Muller,et al.  Application of Boolean algebra to switching circuit design and to error detection , 1954, Trans. I R E Prof. Group Electron. Comput..

[66]  Shu Lin,et al.  Some Results on Cyclic Codes which Are Invariant under the Affine Group and Their Application , 1966, Inf. Control..

[67]  M. Kerimov The theory of error-correcting codes☆ , 1980 .

[68]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[69]  J. Bourgain,et al.  Influences of Variables and Threshold Intervals under Group Symmetries , 1997 .

[70]  Rüdiger L. Urbanke,et al.  Spatially Coupled Ensembles Universally Achieve Capacity Under Belief Propagation , 2013, IEEE Trans. Inf. Theory.

[71]  F. MacWilliams,et al.  The Theory of Error-Correcting Codes , 1977 .

[72]  Andrea Montanari,et al.  The Generalized Area Theorem and Some of its Consequences , 2005, IEEE Transactions on Information Theory.

[73]  Erdal Arikan,et al.  A survey of reed-muller codes from polar coding perspective , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[74]  A. Robert Calderbank,et al.  Beyond double transitivity: Capacity-achieving cyclic codes on erasure channels , 2016, 2016 IEEE Information Theory Workshop (ITW).

[75]  Rudolf Ahlswede,et al.  Good codes can be produced by a few permutations , 1982, IEEE Trans. Inf. Theory.

[76]  Rüdiger L. Urbanke,et al.  Threshold Saturation via Spatial Coupling: Why Convolutional LDPC Ensembles Perform So Well over the BEC , 2010, IEEE Transactions on Information Theory.

[77]  Gilles Zémor,et al.  Discrete Isoperimetric Inequalities and the Probability of a Decoding Error , 2000, Combinatorics, Probability and Computing.

[78]  Rodney M. Goodman,et al.  Any code of which we cannot think is good , 1990, IEEE Trans. Inf. Theory.

[79]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[80]  Raphael Rossignol Threshold for monotone symmetric properties through a logarithmic Sobolev inequality , 2005, math/0511607.

[81]  Christopher Umans,et al.  Simple extractors for all min-entropies and a new pseudo-random generator , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[82]  Daniel A. Spielman Linear-time encodable and decodable error-correcting codes , 1996, IEEE Trans. Inf. Theory.

[83]  Shmuel Safra,et al.  Threshold Phenomena and Influence, with Some Perspectives from Mathematics, Computer Science, and Economics , 2005 .

[84]  Michael Horstein,et al.  Review of 'Low-Density Parity-Check Codes' (Gallager, R. G.; 1963) , 1964, IEEE Transactions on Information Theory.

[85]  Rudiger Urbanke,et al.  From Polar to Reed-Muller Codes: A Technique to Improve the Finite-Length Performance , 2014, IEEE Trans. Commun..

[86]  Ramprasad Saptharishi,et al.  Efficiently Decoding Reed-Muller Codes From Random Errors , 2017, IEEE Trans. Inf. Theory.

[87]  Jean-Marie Goethals,et al.  On Generalized Reed-Muller Codes and Their Relatives , 1970, Inf. Control..

[88]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[89]  Nathan Linial,et al.  Collective Coin Flipping , 1989, Adv. Comput. Res..

[90]  Avi Wigderson,et al.  Reed–Muller Codes for Random Erasures and Errors , 2014, IEEE Transactions on Information Theory.

[91]  Santhosh Kumar,et al.  Comparing the bit-MAP and block-MAP decoding thresholds of reed-muller codes on BMS channels , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[92]  Santhosh Kumar,et al.  Reed-Muller Codes Achieve Capacity on Erasure Channels , 2017, IEEE Trans. Inf. Theory.

[93]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[94]  Richard E. Blahut The Gleason-Prange theorem , 1991, IEEE Trans. Inf. Theory.

[95]  Sergey Yekhanin,et al.  Locally Decodable Codes , 2012, Found. Trends Theor. Comput. Sci..

[96]  H. Loeliger,et al.  Algebraic Coding for Iterative Decoding , 2022 .

[97]  E. Friedgut,et al.  Sharp thresholds of graph properties, and the -sat problem , 1999 .

[98]  Nathan Linial,et al.  The influence of variables on Boolean functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[99]  M. Talagrand On Russo's Approximate Zero-One Law , 1994 .

[100]  Madhu Sudan,et al.  2-Transitivity Is Insufficient for Local Testability , 2008, Computational Complexity Conference.