Approaching maximum likelihood performance of LDPC codes by stochastic resonance in noisy iterative decoders

In the 1960s-70s, Taylor and Kuznetsov obtained a remarkable result that information can be reliably retrieved from a noisy channel even if a decoder is made of noisy components. The results of Vasic and Chilappagari presented at the ITA Workshop ten years ago have revived the interest in decoders made of noisy hardware and since then a number of improvements of the iterative decoders have been made to bring their performance closer to that of their perfect counterparts. However, a common mantra has been that noisy decoders cannot be better than their perfect counterparts. In this talk we report an unexpected phenomenon we have recently discovered — noise can actually improve the error correction process by reducing the probability of decoding error, in some cases by more that two orders of magnitude. This new form of stochastic resonance enables us to use logic gate errors to correct channel errors. This novelty recognizes that the decoder — essentially an iterative minimization of the Bethe free energy on the code graph — can get trapped in local minima, and random perturbations help the decoder to escape from these minima and converge to a correct code-word. In the spirit of Marcus Tullius Cicero's “Clavus clavo eicitur,” (“one nail drives out another”) they operate on the principle: Error errore eicitur” — “one error drives out another.” Crucially, such useful random perturbations require neither additional hardware nor energy, as they are built into the low-powered, noisy hardware itself.

[1]  Shashi Kiran Chilappagari,et al.  An Information Theoretical Framework for Analysis and Design of Nanoscale Fault-Tolerant Memories Based on Low-Density Parity-Check Codes , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  Amin Karbasi,et al.  Noise-Enhanced Associative Memories , 2013, NIPS.

[3]  B. Vasic,et al.  Fault Tolerant Memories Based on Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[4]  Shie Mannor,et al.  Dithered Belief Propagation Decoding , 2012, IEEE Transactions on Communications.

[5]  David Declercq,et al.  Finite Alphabet Iterative Decoders, Part II: Improved Guaranteed Error Correction of LDPC Codes via Iterative Decoder Diversity , 2012, ArXiv.

[6]  Ichi Takumi,et al.  Gradient descent bit flipping algorithms for decoding LDPC codes , 2007, IEEE Transactions on Communications.

[7]  Michael G. Taylor Reliable information storage in memories designed from unreliable components , 1968 .

[8]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[9]  R. M. Tanner,et al.  A Class of Group-Structured LDPC Codes , 2001 .

[10]  David Burshtein,et al.  Expander graph arguments for message-passing algorithms , 2001, IEEE Trans. Inf. Theory.

[11]  B. Vasic,et al.  Trapping set ontology , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[12]  David Declercq,et al.  Finite Alphabet Iterative Decoders—Part II: Towards Guaranteed Error Correction of LDPC Codes via Iterative Decoder Diversity , 2012, IEEE Transactions on Communications.

[13]  David Declercq,et al.  Approaching maximum likelihood decoding of finite length LDPC codes via FAID diversity , 2012, 2012 IEEE Information Theory Workshop.

[14]  Shashi Kiran Chilappagari,et al.  Failures of the Gallager B Decoder: Analysis and Applications , 2006 .

[15]  David Declercq,et al.  Finite alphabet iterative decoders approaching maximum likelihood performance on the Binary Symmetric Channel , 2012, 2012 Information Theory and Applications Workshop.

[16]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[17]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[18]  Bane V. Vasic,et al.  MUDRI: A fault-tolerant decoding algorithm , 2015, 2015 IEEE International Conference on Communications (ICC).

[19]  J. Alvin Connelly,et al.  A noise-based IC random number generator for applications in cryptography , 2000 .

[20]  Shashi Kiran Chilappagari,et al.  Error-Correction Capability of Column-Weight-Three LDPC Codes , 2007, IEEE Transactions on Information Theory.

[21]  Alexios Balatsoukas-Stimming,et al.  Density Evolution for Min-Sum Decoding of LDPC Codes Under Unreliable Message Storage , 2014, IEEE Communications Letters.

[22]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[23]  V Chernyak,et al.  Error correction on a tree: an instanton approach. , 2004, Physical review letters.

[24]  David Declercq,et al.  Finite Alphabet Iterative Decoders—Part I: Decoding Beyond Belief Propagation on the Binary Symmetric Channel , 2013, IEEE Transactions on Communications.

[25]  Nenad Miladinovic,et al.  Improved bit-flipping decoding of low-density parity-check codes , 2002, IEEE Transactions on Information Theory.

[26]  Emmanuel Boutillon,et al.  Noisy Gradient Descent Bit-Flip Decoding for LDPC Codes , 2014, IEEE Transactions on Communications.

[27]  Hilbert J. Kappen,et al.  Sufficient Conditions for Convergence of the Sum–Product Algorithm , 2005, IEEE Transactions on Information Theory.

[28]  Bane V. Vasic,et al.  Fault-Tolerant Probabilistic Gradient-Descent Bit Flipping Decoder , 2014, IEEE Communications Letters.

[29]  Lara Dolecek,et al.  Gallager B Decoder on Noisy Hardware , 2013, IEEE Transactions on Communications.

[30]  Bane V. Vasic,et al.  Analysis of one-step majority logic decoding under correlated data-dependent gate failures , 2014, 2014 IEEE International Symposium on Information Theory.

[31]  Shashi Kiran Chilappagari,et al.  Failures and Error Floors of Iterative Decoders , 2014 .