A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions

Low-density parity-check (LDPC) convolutional codes (or spatially coupled codes) were recently shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is now called threshold saturation via spatial coupling. This new phenomenon is characterized by the belief-propagation threshold of the spatially coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a wide class of coupled scalar recursions. Our approach is based on constructing potential functions for both the coupled and uncoupled recursions. Our results actually show that the fixed point of the coupled recursion is essentially determined by the minimum of the uncoupled potential function and we refer to this phenomenon as Maxwell saturation. A variety of examples are considered including the density-evolution equations for: irregular LDPC codes on the BEC, irregular low-density generator-matrix codes on the BEC, a class of generalized LDPC codes with BCH component codes, the joint iterative decoding of LDPC codes on intersymbol-interference channels with erasure noise, and the compressed sensing of random vectors with independent identically distributed components.

[1]  Ronald F. Boisvert,et al.  NIST Handbook of Mathematical Functions , 2010 .

[2]  Gerald B. Folland,et al.  Real Analysis: Modern Techniques and Their Applications , 1984 .

[3]  Gerhard Fettweis,et al.  On the thresholds of generalized LDPC convolutional codes based on protographs , 2010, 2010 IEEE International Symposium on Information Theory.

[4]  Giulio Colavolpe,et al.  Nonbinary spatially-coupled LDPC codes on the binary erasure channel , 2013, 2013 IEEE International Conference on Communications (ICC).

[5]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[6]  Dmitri V. Truhachev.  Achieving AWGN Multiple Access Channel Capacity with Spatial Graph Coupling , 2012, IEEE Communications Letters.

[7]  Nicolas Macris,et al.  Spatial Coupling as a Proof Technique , 2013, ArXiv.

[8]  Toshiyuki Tanaka,et al.  Improvement of BP-based CDMA multiuser detection by spatial coupling , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[9]  Christian Schlegel,et al.  Multiple Access Demodulation in the Lifted Signal Graph With Spatial Coupling , 2011, IEEE Transactions on Information Theory.

[10]  Kenta Kasai,et al.  Spatially-coupled MacKay-Neal codes and Hsu-Anastasopoulos codes , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[11]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[12]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[13]  Brian M. Kurkoski,et al.  Threshold improvement of low-density lattice codes via spatial coupling , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[14]  Kenta Kasai,et al.  Spatially-coupled multi-edge type LDPC codes with bounded degrees that achieve capacity on the BEC under BP decoding , 2013, 2013 IEEE International Symposium on Information Theory.

[15]  Giuseppe Caire,et al.  Iterative multiuser joint decoding: Unified framework and asymptotic analysis , 2002, IEEE Trans. Inf. Theory.

[16]  Nicolas Macris,et al.  Coupled graphical models and their thresholds , 2010, 2010 IEEE Information Theory Workshop.

[17]  Rüdiger L. Urbanke,et al.  Wave-Like Solutions of General One-Dimensional Spatially Coupled Systems , 2012, ArXiv.

[18]  Michael Lentmaier,et al.  Terminated LDPC convolutional codes with thresholds close to capacity , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[19]  Christian Schlegel,et al.  Thresholds of spatially coupled systems via Lyapunov's method , 2013, 2013 IEEE Information Theory Workshop (ITW).

[20]  Rüdiger L. Urbanke,et al.  Spatially coupled ensembles universally achieve capacity under belief propagation , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[21]  Hideki Imai,et al.  Spatially coupled quasi-cyclic quantum LDPC codes , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[22]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[23]  Henry D. Pfister,et al.  Advanced coding techniques with applications to storage systems , 2012 .

[24]  Henry D. Pfister,et al.  Universality for the noisy Slepian-Wolf problem via spatial coupling , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[25]  Florent Krzakala,et al.  Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices , 2012, ArXiv.

[26]  Andrea Montanari,et al.  The Generalized Area Theorem and Some of its Consequences , 2005, IEEE Transactions on Information Theory.

[27]  Nicolas Macris,et al.  Lossy source coding via spatially coupled LDGM ensembles , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[28]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[29]  Nicolas Macris,et al.  Approaching the rate-distortion limit by spatial coupling with belief propagation and decimation , 2013, 2013 IEEE International Symposium on Information Theory.

[30]  Nicolas Macris,et al.  Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes , 2008, IEEE Transactions on Information Theory.

[31]  Michael Lentmaier,et al.  Iterative Decoding Threshold Analysis for LDPC Convolutional Codes , 2010, IEEE Transactions on Information Theory.

[32]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[33]  Henry D. Pfister,et al.  A simple proof of threshold saturation for coupled scalar recursions , 2012, 2012 7th International Symposium on Turbo Codes and Iterative Information Processing (ISTC).

[34]  Nicolas Macris,et al.  Chains of mean-field models , 2011, ArXiv.

[35]  Nicolas Macris,et al.  A proof of threshold saturation for spatially-coupled LDPC codes on BMS channels , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[36]  Kenta Kasai,et al.  Spatially coupled LDPC codes for decode-and-forward in erasure relay channel , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[37]  Jessica J. Fridrich,et al.  Binary quantization using Belief Propagation with decimation over factor graphs of LDGM codes , 2007, ArXiv.

[38]  Shlomo Shamai,et al.  Estimation in Gaussian Noise: Properties of the Minimum Mean-Square Error , 2010, IEEE Transactions on Information Theory.

[39]  Rüdiger L. Urbanke,et al.  Threshold saturation on BMS channels via spatial coupling , 2010, 2010 6th International Symposium on Turbo Codes & Iterative Information Processing.

[40]  Andrea Montanari,et al.  Asymptotic Rate versus Design Rate , 2007, 2007 IEEE International Symposium on Information Theory.

[41]  Nicolas Macris,et al.  And now to something completely different: Spatial coupling as a proof technique , 2013, 2013 IEEE International Symposium on Information Theory.

[42]  Michael Lentmaier,et al.  Convergence analysis for a class of LDPC convolutional codes on the erasure channel , 2004 .

[43]  Paul H. Siegel,et al.  Joint iterative decoding of LDPC codes for channels with memory and erasure noise , 2008, IEEE Journal on Selected Areas in Communications.

[44]  Florent Krzakala,et al.  Statistical physics-based reconstruction in compressed sensing , 2011, ArXiv.

[45]  Henry D. Pfister,et al.  A simple proof of threshold saturation for coupled vector recursions , 2012, 2012 IEEE Information Theory Workshop.

[46]  Henry D. Pfister,et al.  Approaching capacity at high rates with iterative hard-decision decoding , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[47]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[48]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[49]  Henry D. Pfister,et al.  The effect of spatial coupling on compressive sensing , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[50]  Henry D. Pfister,et al.  Threshold saturation of spatially-coupled codes on intersymbol-interference channels , 2012, 2012 IEEE International Conference on Communications (ICC).

[51]  Paul R. Milgrom,et al.  Envelope Theorems for Arbitrary Choice Sets , 2002 .

[52]  Nicolas Macris,et al.  Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems , 2011, ArXiv.

[53]  Rüdiger L. Urbanke,et al.  Threshold Saturation via Spatial Coupling: Why Convolutional LDPC Ensembles Perform So Well over the BEC , 2010, IEEE Transactions on Information Theory.

[54]  Henry D. Pfister,et al.  Spatially-Coupled Codes and Threshold Saturation on Intersymbol-Interference Channels , 2011, ArXiv.

[55]  Kenta Kasai,et al.  Spatially coupled codes over the multiple access channel , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[56]  Adel Javanmard,et al.  Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing , 2011, IEEE Transactions on Information Theory.

[57]  Mikael Skoglund,et al.  Rate-equivocation optimal spatially coupled LDPC codes for the BEC wiretap channel , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[58]  Dongning Guo,et al.  A single-letter characterization of optimal noisy compressed sensing , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[59]  Toshiyuki Tanaka,et al.  A Phenomenological Study on Threshold Improvement via Spatial Coupling , 2012, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[60]  Kamil Sh. Zigangirov,et al.  Time-varying periodic convolutional codes with low-density parity-check matrix , 1999, IEEE Trans. Inf. Theory.

[61]  Nicolas Macris,et al.  Threshold Saturation for Spatially Coupled LDPC and LDGM Codes on BMS Channels , 2013, IEEE Transactions on Information Theory.

[62]  Henry D. Pfister,et al.  Universal codes for the Gaussian MAC via spatial coupling , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[63]  Kenta Kasai,et al.  Threshold saturation on channels with memory via spatial coupling , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.