Wave-Like Solutions of General One-Dimensional Spatially Coupled Systems

We establish the existence of wave-like solutions to spatially coupled graphical models which, in the large size limit, can be characterized by a one-dimensional real-valued state. This is extended to a proof of the threshold saturation phenomenon for all such models, which includes spatially coupled irregular LDPC codes over the BEC, but also addresses hard-decision decoding for transmission over general channels, the CDMA multiple-access problem, compressed sensing, and some statistical physics models. For traditional uncoupled iterative coding systems with two components and transmission over the BEC, the asymptotic convergence behavior is completely characterized by the EXIT curves of the components. More precisely, the system converges to the desired fixed point, which is the one corresponding to perfect decoding, if and only if the two EXIT functions describing the components do not cross. For spatially coupled systems whose state is one-dimensional a closely related graphical criterion applies. Now the curves are allowed to cross, but not by too much. More precisely, we show that the threshold saturation phenomenon is related to the positivity of the (signed) area enclosed by two EXIT-like functions associated to the component systems, a very intuitive and easy-to-use graphical characterization. In the spirit of EXIT functions and Gaussian approximations, we also show how to apply the technique to higher dimensional and even infinite-dimensional cases. In these scenarios the method is no longer rigorous, but it typically gives accurate predictions. To demonstrate this application, we discuss transmission over general channels using both the belief-propagation as well as the min-sum decoder.

[1]  Toshiyuki Tanaka,et al.  A Phenomenological Study on Threshold Improvement via Spatial Coupling , 2012, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[2]  Kamil Sh. Zigangirov,et al.  Time-varying periodic convolutional codes with low-density parity-check matrix , 1999, IEEE Trans. Inf. Theory.

[3]  Rüdiger L. Urbanke,et al.  Threshold Saturation via Spatial Coupling: Why Convolutional LDPC Ensembles Perform So Well over the BEC , 2010, IEEE Transactions on Information Theory.

[4]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[5]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[6]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[7]  Toshiyuki Tanaka,et al.  Improvement of BP-based CDMA multiuser detection by spatial coupling , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[8]  Henry D. Pfister,et al.  The effect of spatial coupling on compressive sensing , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  Nicolas Macris,et al.  Analysis of coupled scalar systems by displacement convexity , 2014, 2014 IEEE International Symposium on Information Theory.

[10]  Stephan ten Brink,et al.  Convergence behavior of iteratively decoded parallel concatenated codes , 2001, IEEE Trans. Commun..

[11]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[12]  Michael Lentmaier,et al.  To the Theory of Low-Density Convolutional Codes. II , 2001, Probl. Inf. Transm..

[13]  Michael Lentmaier,et al.  Convergence analysis for a class of LDPC convolutional codes on the erasure channel , 2004 .

[14]  Florent Krzakala,et al.  Statistical physics-based reconstruction in compressed sensing , 2011, ArXiv.

[15]  Henry D. Pfister,et al.  A simple proof of threshold saturation for coupled vector recursions , 2012, 2012 IEEE Information Theory Workshop.

[16]  Christian Schlegel,et al.  Multiple Access Demodulation in the Lifted Signal Graph With Spatial Coupling , 2011, IEEE Transactions on Information Theory.

[17]  Sae-Young Chung,et al.  On the construction of some capacity-approaching coding schemes , 2000 .

[18]  C. Méasson Conservation laws for coding , 2006 .

[19]  Rüdiger L. Urbanke,et al.  Exact thresholds and optimal codes for the binary-symmetric channel and Gallager's decoding algorithm A , 2000, IEEE Transactions on Information Theory.

[20]  Toshiyuki Tanaka,et al.  A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors , 2002, IEEE Trans. Inf. Theory.

[21]  Dmitri V. Truhachev,et al.  Universal multiple access via spatially coupling data transmission , 2013, 2013 IEEE International Symposium on Information Theory.

[22]  Yihong Wu,et al.  Rényi Information Dimension: Fundamental Limits of Almost Lossless Analog Compression , 2010, IEEE Transactions on Information Theory.

[23]  Michael Lentmaier,et al.  Terminated LDPC convolutional codes with thresholds close to capacity , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[24]  Rüdiger L. Urbanke,et al.  Spatially coupled ensembles universally achieve capacity under belief propagation , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[25]  Henry D. Pfister,et al.  A simple proof of threshold saturation for coupled scalar recursions , 2012, 2012 7th International Symposium on Turbo Codes and Iterative Information Processing (ISTC).

[26]  Adel Javanmard,et al.  Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing , 2011, IEEE Transactions on Information Theory.

[27]  Nicolas Macris,et al.  Chains of mean-field models , 2011, ArXiv.

[28]  Nicolas Macris,et al.  How to prove the Maxwell conjecture via spatial coupling — A proof of concept , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[29]  Michael Lentmaier,et al.  Iterative Decoding Threshold Analysis for LDPC Convolutional Codes , 2010, IEEE Transactions on Information Theory.

[30]  Michael Lentmaier,et al.  On the Theory of Low-Density Convolutional Codes , 1999, AAECC.

[31]  N. Rose,et al.  Differential Equations With Applications , 1967 .

[32]  Daniel J. Costello,et al.  LDPC block and convolutional codes based on circulant matrices , 2004, IEEE Transactions on Information Theory.

[33]  Henry D. Pfister,et al.  Universality for the noisy Slepian-Wolf problem via spatial coupling , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[34]  S. Brink Convergence of iterative decoding , 1999 .

[35]  Stephan ten Brink Iterative Decoding Trajectories of Parallel Concatenated Codes , 1999 .