Iterative Decoding of Codes on Graphs

The growing popularity of a class of linear block codes called the low-density parity-check (LDPC) codes can be attributed to the low complexity of the iterative decoders, and their potential to achieve performance very close to the Shannon capacity. This makes them an attractive candidate for ECC applications in communication systems. This report proposes methods to systematically construct regular and irregular LDPC codes.A class of regular LDPC codes are constructed from incidence structures in finite geometries like projective geometry and affine geometry. A class of irregular LDPC codes are constructed by systematically splitting blocks of balanced incomplete block designs to achieve desired weight distributions. These codes are decoded iteratively using message-passing algorithms, and the performance of these codes for various channels are presented in this report.The application of iterative decoders is generally limited to a class of codes whose graph representations are free of small cycles. Unfortunately, the large class of conventional algebraic codes, like RS codes, has several four cycles in their graph representations. This report proposes an algorithm that aims to alleviate this drawback by constructing an equivalent graph representation that is free of four cycles. It is theoretically shown that the four-cycle free representation is better suited to iterative erasure decoding than the conventional representation. Also, the new representation is exploited to realize, with limited success, iterative decoding of Reed-Solomon codes over the additive white Gaussian noise channel.Wiberg, Forney, Richardson, Koetter, and Vontobel have made significant contributions in developing theoretical frameworks that facilitate finite length analysis of codes. With an exception of Richardson's, most of the other frameworks are much suited for the analysis of short codes. In this report, we further the understanding of the failures in iterative decoders for the binary symmetric channel. The failures of the decoder are classified into two categories by defining trapping sets and propagating sets. Such a classification leads to a successful estimation of the performance of codes under the Gallager B decoder. Especially, the estimation techniques show great promise in the high signal-to-noise ratio regime where the simulation techniques are less feasible.

[1]  Andrew J. Viterbi,et al.  An Intuitive Justification and a Simplified Implementation of the MAP Decoder for Convolutional Codes , 1998, IEEE J. Sel. Areas Commun..

[2]  Hans-Andrea Loeliger,et al.  Codes and iterative decoding on general graphs , 1995, Eur. Trans. Telecommun..

[3]  Jing Li,et al.  On the performance of high-rate TPC/SPC codes and LDPC codes over partial response channels , 2002, IEEE Trans. Commun..

[4]  Jennifer D. Key,et al.  Designs and their codes , 1992, Cambridge tracts in mathematics.

[5]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[6]  I. Djordjevic,et al.  High code rate low-density parity-check codes for optical communication systems , 2004, IEEE Photonics Technology Letters.

[7]  Hanfried Lenz,et al.  Design theory , 1985 .

[8]  Joachim Rosenthal,et al.  Pseudocodeword weights and stopping sets , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[9]  Robert J. McEliece,et al.  On the BCJR trellis for linear block codes , 1996, IEEE Trans. Inf. Theory.

[10]  Bane Vasic,et al.  A systematic construction of irregular low-density parity-check codes from combinatorial designs , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[11]  I.B. Djordjevic,et al.  Regular and irregular low-density parity-check codes for ultra-long haul high-speed optical communications: construction and performance analysis , 2004, Optical Fiber Communication Conference, 2004. OFC 2004.

[12]  Xiao Ma,et al.  Binary intersymbol interference channels: Gallager codes, density evolution, and code performance bounds , 2003, IEEE Transactions on Information Theory.

[13]  D.J.C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[14]  I.B. Djordjevic,et al.  Block-circulant low-density parity-check codes for optical communication systems , 2004, IEEE Journal of Selected Topics in Quantum Electronics.

[15]  R. Koetter,et al.  On the Effective Weights of Pseudocodewords for Codes Defined on Graphs with Cycles , 2001 .

[16]  Evangelos Eleftheriou,et al.  Performance of product codes on channels with memory , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[17]  Madhu Sudan Coding Theory: Tutorial and Survey , 2001, FOCS 2001.

[18]  Jan W. M. Bergmans,et al.  Digital baseband transmission and recording , 1996 .

[19]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .

[20]  Brendan J. Frey,et al.  Signal-space characterization of iterative decoding , 2001, IEEE Trans. Inf. Theory.

[21]  Bane Vasic,et al.  Lattice low-density parity check codes and their application in partial response systems , 2002, Proceedings IEEE International Symposium on Information Theory,.

[22]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[23]  Jung-Fu Cheng Iterative decoding , 1998 .

[24]  V Chernyak,et al.  Error correction on a tree: an instanton approach. , 2004, Physical review letters.

[25]  Emre Telatar,et al.  Finite-length analysis of low-density parity-check codes on the binary erasure channel , 2002, IEEE Trans. Inf. Theory.

[26]  Robert Michael Tanner,et al.  Minimum-distance bounds by graph analysis , 2001, IEEE Trans. Inf. Theory.

[27]  Bane Vasic,et al.  Irregular low-density parity-check codes: construction and performance on perpendicular magnetic recording channels , 2003 .

[28]  S. W. McLaughlin,et al.  Iterative decoding for partial response (PR), equalized, magneto-optical (MO) data storage channels , 2001, IEEE J. Sel. Areas Commun..

[29]  R. M. Tanner,et al.  A Class of Group-Structured LDPC Codes , 2001 .

[30]  Thomas J. Richardson,et al.  Error Floors of LDPC Codes , 2003 .

[31]  Mario Blaum An Introduction to Error- Correcting Codes , 2004 .

[32]  Bane V. Vasic,et al.  Iterative decoding of linear block codes: a parity-check orthogonalization approach , 2005, IEEE Transactions on Information Theory.

[33]  Shu Lin,et al.  Low-density parity-check codes based on finite geometries: A rediscovery and new results , 2001, IEEE Trans. Inf. Theory.

[34]  Bane V. Vasic,et al.  Combinatorial constructions of low-density parity-check codes for iterative decoding , 2002, IEEE Transactions on Information Theory.

[35]  Bane V. Vasic,et al.  Structured iteratively decodable codes based on Steiner systems and their application in magnetic recording , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[36]  B. Vasic,et al.  Iteratively decodable codes on m flats for WDM high-speed long-haul transmission , 2005, Journal of Lightwave Technology.

[37]  J. D. Key,et al.  Designs and codes: An update , 1996 .

[38]  Ramesh Pyndiah,et al.  Near-optimum decoding of product codes: block turbo codes , 1998, IEEE Trans. Commun..

[39]  Bane Vasic,et al.  LDPC codes based on mutually orthogonal latin rectangles and their application in perpendicular magnetic recording , 2002 .

[40]  Alexander Graham,et al.  Kronecker Products and Matrix Calculus: With Applications , 1981 .

[41]  Francis Buekenhout Handbook of incidence geometry : buildings and foundations , 1995 .

[42]  Amin Shokrollahi,et al.  LDPC Codes: An Introduction , 2004 .

[43]  J. D. Key,et al.  Baer Subplanes, Ovals and Unitals , 1990 .

[44]  T. Etzion,et al.  Which codes have cycle-free Tanner graphs? , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[45]  D. Spielman,et al.  Expander codes , 1996 .

[46]  Niclas Wiberg,et al.  Codes and Decoding on General Graphs , 1996 .

[47]  Ajay Dholakia,et al.  Efficient implementations of the sum-product algorithm for decoding LDPC codes , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[48]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[49]  Jennifer D. Key,et al.  Designs and Codes: An update , 1996, Des. Codes Cryptogr..

[50]  S. Sankaranarayanan,et al.  Projective-plane iteratively decodable block codes for WDM high-speed long-haul transmission systems , 2004, Journal of Lightwave Technology.

[51]  Rüdiger L. Urbanke,et al.  Design of capacity-approaching irregular low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[52]  Evelyne Contejean,et al.  An Efficient Incremental Algorithm for Solving Systems of Linear Diophantine Equations , 1994, Inf. Comput..

[53]  C. Colbourn,et al.  Handbook of Combinatorial Designs , 2006 .

[54]  Alexander Vardy,et al.  Algebraic soft-decision decoding of Reed-Solomon codes , 2003, IEEE Trans. Inf. Theory.

[55]  Noga Alon,et al.  Finding and counting given length cycles , 1997, Algorithmica.

[56]  J. H. van Lint,et al.  Introduction to Coding Theory , 1982 .

[57]  E. Gilbert Capacity of a burst-noise channel , 1960 .

[58]  Keith M. Chugg,et al.  An algorithm for counting short cycles in bipartite graphs , 2006, IEEE Transactions on Information Theory.

[59]  Erozan M. Kurtas,et al.  Signal and Noise Generation for Magnetic Recording Channel Simulations , 2004 .

[60]  P. Vontobel,et al.  Constructions of regular and irregular LDPC codes using Ramanujan graphs and ideas from Margulis , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[61]  E. O. Elliott Estimates of error rates for codes on burst-noise channels , 1963 .

[62]  P. Vontobel,et al.  Graph-Cover Decoding and Finite-Length Analysis of Message-Passing Iterative Decoding of LDPC Codes , 2005, ArXiv.

[63]  Shu Lin,et al.  A combinatoric superposition method for constructing low density parity check codes , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[64]  Joachim Hagenauer,et al.  Iterative decoding of binary block and convolutional codes , 1996, IEEE Trans. Inf. Theory.

[65]  N. Hamada,et al.  On the $p$-rank of the incidence matrix of a balanced or partially balanced incomplete block design and its applications to error correcting codes , 1973 .

[66]  Alexander Vardy,et al.  Bit-level soft-decision decoding of Reed-Solomon codes , 1991, IEEE Trans. Commun..

[67]  C. Colbourn,et al.  The CRC handbook of combinatorial designs , edited by Charles J. Colbourn and Jeffrey H. Dinitz. Pp. 784. $89.95. 1996. ISBN 0-8493-8948-8 (CRC). , 1997, The Mathematical Gazette.

[68]  Luther D. Rudolph,et al.  A class of majority logic decodable codes (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[69]  I.B. Djordjevic,et al.  Low-density parity-check codes for 40-gb/s optical transmission systems , 2006, IEEE Journal of Selected Topics in Quantum Electronics.

[70]  Robert Michael Tanner,et al.  A recursive approach to low complexity codes , 1981, IEEE Trans. Inf. Theory.

[71]  Jaekyun Moon Modeling the Recording Channel , 2004 .

[72]  G. Nemhauser,et al.  Integer Programming , 2020 .

[73]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[74]  David J. C. MacKay,et al.  Weaknesses of Margulis and Ramanujan-Margulis low-density parity-check cCodes , 2003, MFCSIT.

[75]  Eric Domenjoud,et al.  Solving Systems of Linear Diophantine Equations: An Algebraic Approach , 1991, MFCS.

[76]  A. Robert Calderbank,et al.  Minimal tail-biting trellises: The Golay code and more , 1999, IEEE Trans. Inf. Theory.

[77]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[78]  Dwijendra K. Ray-Chaudhuri,et al.  Binary mixture flow with free energy lattice Boltzmann methods , 2022, arXiv.org.

[79]  Shashi Kiran Chilappagari,et al.  Error Floors of LDPC Codes on the Binary Symmetric Channel , 2006, 2006 IEEE International Conference on Communications.

[80]  E.M. Kurtas,et al.  The search for a practical iterative detector for magnetic recording , 2004, IEEE Transactions on Magnetics.

[81]  John Proakis Partial Response Equalization with Application to High Density Magnetic Recording Channels , 2004 .

[82]  Shu Lin,et al.  Codes on finite geometries , 2005, IEEE Transactions on Information Theory.

[83]  Shu Lin,et al.  On the construction of a class of majority-logic decodable codes , 1971, IEEE Trans. Inf. Theory.

[84]  Ian F. Blake,et al.  The mathematical theory of coding , 1975 .

[85]  Daniel A. Spielman,et al.  Analysis of low density codes and improved designs using irregular graphs , 1998, STOC '98.

[86]  J. Moon,et al.  Modeling the Lorentzian magnetic recording channel with transition noise , 2001 .

[87]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[88]  John G. Proakis,et al.  Digital Communications , 1983 .

[89]  I.B. Djordjevic,et al.  Irregular low-density parity-check codes for long-haul optical communications , 2004, IEEE Photonics Technology Letters.

[90]  D. Mackay,et al.  Evaluation of Gallager Codes for Short Block Length and High Rate Applications , 2001 .

[91]  I.B. Djordjevic,et al.  Projective geometry LDPC codes for ultralong-haul WDM high-speed transmission , 2003, IEEE Photonics Technology Letters.

[92]  Claude Kirchner,et al.  A Methodological View of Constraint Solving , 1999, Constraints.

[93]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[94]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[95]  William E. Ryan A Turbo Code Tutorial , 1997 .

[96]  Jack K. Wolf ECC performance of interleaved RS codes with burst errors , 1998 .

[97]  Sarah J. Johnson,et al.  Regular low-density parity-check codes from oval designs , 2003, Eur. Trans. Telecommun..

[98]  I. Anderson Combinatorial Designs: Construction Methods , 1990 .

[99]  Jinghu Chen,et al.  Generating Code Representations Suitable for Belief Propagation Decoding , 2002 .

[100]  Roy D. Cideciyan,et al.  Perpendicular and longitudinal recording: a signal-processing and coding perspective , 2002 .