Performance of LDPC Codes Under Faulty Iterative Decoding

Departing from traditional communication theory where decoding algorithms are assumed to perform without error, a system where noise perturbs both computational devices and communication channels is considered here. This paper studies limits in processing noisy signals with noisy circuits by investigating the effect of noise on standard iterative decoders for low-density parity-check (LDPC) codes. Concentration of decoding performance around its average is shown to hold when noise is introduced into message-passing and local computation. Density evolution equations for simple faulty iterative decoders are derived. In one model, computing nonlinear estimation thresholds shows that performance degrades smoothly as decoder noise increases, but arbitrarily small probability of error is not achievable. Probability of error may be driven to zero in another system model; the decoding threshold again decreases smoothly with decoder noise. As an application of the methods developed, an achievability result for reliable memory systems constructed from unreliable components is provided.

[1]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[2]  John W. Fisher,et al.  Loopy Belief Propagation: Convergence and Effects of Message Errors , 2005, J. Mach. Learn. Res..

[3]  Anna Gál,et al.  Lower bounds for the complexity of reliable Boolean circuits with noisy gates , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[4]  Rüdiger L. Urbanke,et al.  Exact thresholds and optimal codes for the binary-symmetric channel and Gallager's decoding algorithm A , 2000, IEEE Transactions on Information Theory.

[5]  Naresh R. Shanbhag,et al.  Toward achieving energy efficiency in presence of deep submicron noise , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[6]  Klaus Krickeberg,et al.  Probability and information theory II , 1969 .

[7]  Sergio Telles Ribeiro,et al.  Random-Pulse Machines , 1967, IEEE Trans. Electron. Comput..

[8]  A. Blanksby,et al.  A 690-mW 1-Gb/s 1024-b, rate-1/2 low-density parity-check code decoder , 2001, IEEE J. Solid State Circuits.

[9]  Rüdiger L. Urbanke,et al.  Capacity-achieving ensembles for the binary erasure channel with bounded complexity , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[10]  Christoforos N. Hadjicostis,et al.  Coding Approaches to Fault Tolerance in Combinational and Dynamic Systems , 2001, The Kluwer international series in engineering and computer science.

[11]  Rüdiger L. Urbanke,et al.  Weight Distribution of Low-Density Parity-Check Codes , 2006, IEEE Transactions on Information Theory.

[12]  Anantha Chandrakasan,et al.  Approximate Signal Processing , 1997, J. VLSI Signal Process..

[13]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[14]  B. Vasic,et al.  Can the storage capacity of memories built from unreliable components be determined? , 2008, 2008 Information Theory and Applications Workshop.

[15]  Sanjukta Bhanja,et al.  Probabilistic Error Modeling for Nano-Domain Logic Circuits , 2009, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[16]  Rudiger Urbanke,et al.  Fixed Points and Stability of Density Evolution , 2004, Commun. Inf. Syst..

[17]  Ken Mai,et al.  The future of wires , 2001, Proc. IEEE.

[18]  R. Gallager Information Theory and Reliable Communication , 1968 .

[19]  Lorena Anghel,et al.  Defects Tolerant Logic Gates for Unreliable Future Nanotechnologies , 2007, IWANN.

[20]  Christer Svensson,et al.  Noise in digital dynamic CMOS circuits , 1994 .

[21]  Andrea Montanari,et al.  Tight bounds for LDPC and LDGM codes under MAP decoding , 2004, IEEE Transactions on Information Theory.

[22]  B. Vasic,et al.  Fault Tolerant Memories Based on Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[23]  Hans-Andrea Loeliger,et al.  Probability propagation and decoding in analog VLSI , 2001, IEEE Trans. Inf. Theory.

[24]  Russell Tessier,et al.  Trading off transient fault tolerance and power consumption in deep submicron (DSM) VLSI circuits , 2004, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[25]  Gianluca Piccinini,et al.  Architectural strategies for low-power VLSI turbo decoders , 2002, IEEE Trans. Very Large Scale Integr. Syst..

[26]  S. Brink Convergence of iterative decoding , 1999 .

[27]  Keith M. Chugg,et al.  On the Error Tolerance of Iterative Decoder Circuitry ( Invited Paper ) , 2008 .

[28]  Sever S Dragomir,et al.  New Inequalities for Convex Functions with Applications for the N-Entropy of a Discrete Random Variable , 1999 .

[29]  Nicholas Pippenger Reliable Computation in the Presence of Noise , 1986 .

[30]  M. Chiani Error Detecting and Error Correcting Codes , 2012 .

[31]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[32]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[33]  Stoddart,et al.  Electronically configurable molecular-based logic gates , 1999, Science.

[34]  Rahul Sarpeshkar,et al.  Analog Versus Digital: Extrapolating from Electronics to Neurobiology , 1998, Neural Computation.

[35]  J.A.B. Fortes,et al.  Bifurcations and fundamental error bounds for fault-tolerant computations , 2005, IEEE Transactions on Nanotechnology.

[36]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[37]  Michael G. Taylor Reliable information storage in memories designed from unreliable components , 1968 .

[38]  Vannevar Bush,et al.  The differential analyzer. A new machine for solving differential equations , 1931 .

[39]  Daniel A. Spielman,et al.  Expander codes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[40]  Claude E. Shannon,et al.  Reliable Circuits Using Less Reliable Relays , 1956 .

[41]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[42]  L.R. Varshney,et al.  Performance of LDPC Codes Under Noisy Message-Passing Decoding , 2007, 2007 IEEE Information Theory Workshop.

[43]  R. Ramaswami,et al.  Book Review: Design and Analysis of Fault-Tolerant Digital Systems , 1990 .

[44]  Sanjoy K. Mitter,et al.  Endcoding complexity versus minimum distance , 2005, IEEE Transactions on Information Theory.

[45]  I. M. Jacobs,et al.  Principles of Communication Engineering , 1965 .

[46]  David S. Slepian The threshold effect in modulation systems that expand bandwidth , 1962, IRE Trans. Inf. Theory.

[47]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[48]  Anantha Chandrakasan,et al.  Energy scalable system design , 2002, IEEE Trans. Very Large Scale Integr. Syst..

[49]  Sae-Young Chung,et al.  Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation , 2001, IEEE Trans. Inf. Theory.

[50]  Lu Yang,et al.  Recent Advances on Determining the Number of Real Roots of Parametric Polynomials , 1999, J. Symb. Comput..

[51]  Sujit Dey,et al.  Evaluating Transient Error Effects in Digital Nanometer Circuits , 2007, IEEE Transactions on Reliability.

[52]  Luca Benini,et al.  Networks on chips - technology and tools , 2006, The Morgan Kaufmann series in systems on silicon.

[53]  Amir H. Banihashemi,et al.  Performance of Belief Propagation for Decoding LDPC Codes in the Presence of Channel Estimation Error , 2007, IEEE Transactions on Communications.

[54]  Konstantin K. Likharev,et al.  Single-electron devices and their applications , 1999, Proc. IEEE.

[55]  Nicholas Pippenger,et al.  Reliable computation by formulas in the presence of noise , 1988, IEEE Trans. Inf. Theory.

[56]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[57]  Tong Zhang,et al.  An FPGA Implementation of-Regular Low-Density Parity-Check Code Decoder , 2003, EURASIP J. Adv. Signal Process..

[58]  Keith M. Chugg,et al.  The Extraction and Complexity Limits of Graphical Models for Linear Codes , 2006, IEEE Transactions on Information Theory.

[59]  N. Fisher,et al.  Probability Inequalities for Sums of Bounded Random Variables , 1994 .

[60]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[61]  Barry W. Johnson Design & analysis of fault tolerant digital systems , 1988 .

[62]  Claude E. Shannon,et al.  A symbolic analysis of relay and switching circuits , 1938, Transactions of the American Institute of Electrical Engineers.

[63]  Mohammad M. Mansour,et al.  A 640-Mb/s 2048-bit programmable LDPC decoder chip , 2006, IEEE Journal of Solid-State Circuits.

[64]  David Burshtein,et al.  Expander graph arguments for message-passing algorithms , 2001, IEEE Trans. Inf. Theory.

[65]  Masoud Ardakani,et al.  A more accurate one-dimensional analysis and design of irregular LDPC codes , 2004, IEEE Transactions on Communications.

[66]  Li Ping,et al.  Decoding low density parity check codes with finite quantization bits , 2000, IEEE Communications Letters.

[67]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[68]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[69]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[70]  A. J. Blanksby,et al.  A 690-mW 1-Gb/s 1024-b, rate-1/2 low-density parity-check code decoder , 2001, IEEE J. Solid State Circuits.

[71]  Emre Telatar,et al.  Mismatched decoding revisited: General alphabets, channels with memory, and the wide-band limit , 2000, IEEE Trans. Inf. Theory.

[72]  Leonard J. Schulman,et al.  Signal propagation and noisy circuits , 1999, IEEE Trans. Inf. Theory.

[73]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[74]  E.,et al.  A. Computation in the Presence of Noise , 2009 .

[75]  R. Singhal,et al.  Quantized LDPC decoder design for binary symmetric channels , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[76]  Kazuoki Azuma WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[77]  Daniel J. Costello,et al.  Channel coding: The road to channel capacity , 2006, Proceedings of the IEEE.

[78]  Jon Feldman,et al.  Nonlinear programming approaches to decoding low-density parity-check codes , 2006, IEEE Journal on Selected Areas in Communications.

[79]  B. Jack Copeland,et al.  Hypercomputation , 2004, Minds and Machines.

[80]  Anantha Chandrakasan,et al.  Quantifying and enhancing power awareness of VLSI systems , 2001, IEEE Trans. Very Large Scale Integr. Syst..

[81]  Aaron D. Wyner,et al.  Reliable Circuits Using Less Reliable Relays , 1993 .

[82]  Ajay Dholakia,et al.  Reduced-complexity decoding of LDPC codes , 2005, IEEE Transactions on Communications.

[83]  Amir H. Banihashemi,et al.  On implementation of min-sum algorithm and its modifications for decoding low-density Parity-check (LDPC) codes , 2005, IEEE Transactions on Communications.

[84]  Kannan Ramchandran,et al.  Network Coding for Distributed Storage in Wireless Networks , 2008 .

[85]  V. Erokhin $\varepsilon $-Entropy of a Discrete Random Variable , 1958 .

[86]  Philip M. Woodward,et al.  Probability and Information Theory with Applications to Radar , 1954 .

[87]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[88]  Sundeep Rangan,et al.  Recursive consistent estimation with bounded noise , 2001, IEEE Trans. Inf. Theory.

[89]  Nenad Miladinovic,et al.  Improved bit-flipping decoding of low-density parity-check codes , 2002, IEEE Transactions on Information Theory.

[90]  Michael Lentmaier,et al.  An analysis of the block error probability performance of iterative decoding , 2005, IEEE Transactions on Information Theory.

[91]  A. Robert Calderbank,et al.  The Art of Signaling: Fifty Years of Coding Theory , 1998, IEEE Trans. Inf. Theory.

[92]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[93]  Elwyn R. Berlekamp,et al.  A lower bound to the distribution of computation for sequential decoding , 1967, IEEE Trans. Inf. Theory.

[94]  Shie Mannor,et al.  Stochastic decoding of LDPC codes , 2006, IEEE Communications Letters.

[95]  Naresh R. Shanbhag,et al.  Energy-efficiency bounds for deep submicron VLSI systems in the presence of noise , 2003, IEEE Trans. Very Large Scale Integr. Syst..

[96]  Anant Sahai,et al.  The price of certainty: "waterslide curves" and the gap to capacity , 2008, ArXiv.

[97]  Tianli Yu,et al.  Efficient Message Representations for Belief Propagation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[98]  P. Jonker,et al.  A defect-?and fault-tolerant architecture for nanocomputers , 2003 .

[99]  M. Wilkes Computer Design , 1961, Nature.

[100]  Rüdiger L. Urbanke,et al.  Design of capacity-approaching irregular low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[101]  Anant Sahai,et al.  The Necessity and Sufficiency of Anytime Capacity for Stabilization of a Linear System Over a Noisy Communication Link—Part I: Scalar Systems , 2006, IEEE Transactions on Information Theory.

[102]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[103]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[104]  J.L. Massey,et al.  Theory and practice of error control codes , 1986, Proceedings of the IEEE.

[105]  P.K. Varshney,et al.  Channel-aware distributed detection in wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[106]  Niraj K. Jha,et al.  Fault-tolerant computer system design , 1996, IEEE Parallel & Distributed Technology: Systems & Applications.

[108]  C. Dekker,et al.  Logic Circuits with Carbon Nanotube Transistors , 2001, Science.

[109]  Sae-Young Chung,et al.  On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit , 2001, IEEE Communications Letters.

[110]  Rüdiger L. Urbanke,et al.  Density Evolution, Thresholds and the Stability Condition for Non-binary LDPC Codes , 2005, ArXiv.

[111]  Avi Pfeffer,et al.  Loopy Belief Propagation as a Basis for Communication in Sensor Networks , 2002, UAI.

[112]  Bernard Widrow,et al.  Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications , 2008 .