Parity-check density versus performance of binary linear block codes over memoryless symmetric channels

We derive lower bounds on the density of parity-check matrices of binary linear codes which are used over memoryless binary-input output-symmetric (MBIOS) channels. The bounds are expressed in terms of the gap between the rate of these codes for which reliable communications is achievable and the channel capacity; they are valid for every sequence of binary linear block codes if there exists a decoding algorithm under which the average bit-error probability vanishes. For every MBIOS channel, we construct a sequence of ensembles of regular low-density parity-check (LDPC) codes, so that an upper bound on the asymptotic density of their parity-check matrices scales similarly to the lower bound. The tightness of the lower bound is demonstrated for the binary erasure channel by analyzing a sequence of ensembles of right-regular LDPC codes which was introduced by Shokrollahi, and which is known to achieve the capacity of this channel. Under iterative message-passing decoding, we show that this sequence of ensembles is asymptotically optimal (in a sense to be defined in this paper), strengthening a result of Shokrollahi. Finally, we derive lower bounds on the bit-error probability and on the gap to capacity for binary linear block codes which are represented by bipartite graphs, and study their performance limitations over MBIOS channels. The latter bounds provide a quantitative measure for the number of cycles of bipartite graphs which represent good error-correction codes.

[1]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[2]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[3]  Elwyn R. Berlekamp,et al.  On the inherent intractability of certain coding problems (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[4]  John T. Coffey,et al.  The capacity of coded systems , 1997, IEEE Trans. Inf. Theory.

[5]  T. Etzion,et al.  Which codes have cycle-free Tanner graphs? , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[6]  J. Gross,et al.  Graph Theory and Its Applications , 1998 .

[7]  Oliver M. Collins,et al.  The Capacity of Binary Channels that Use Linear Codes and Decoders , 1998, IEEE Trans. Inf. Theory.

[8]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[9]  Amin Shokrollahi,et al.  New Sequences of Linear Time Erasure Codes Approaching the Channel Capacity , 1999, AAECC.

[10]  Meir Feder,et al.  Random coding techniques for nonrandom codes , 1999, IEEE Trans. Inf. Theory.

[11]  Robert J. McEliece.  Achieving the Shannon Limit: A Progress Report , 2000 .

[12]  Shlomo Shamai,et al.  Improved upper bounds on the ensemble performance of ML decoded low density parity check codes , 2000, IEEE Communications Letters.

[13]  Rüdiger L. Urbanke,et al.  Design of capacity-approaching irregular low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[14]  Rüdiger L. Urbanke,et al.  Efficient encoding of low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[15]  M. Shokrollahi,et al.  Capacity-achieving sequences , 2001 .

[16]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[17]  Robert J. McEliece,et al.  On the complexity of reliable communication on the erasure channel , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[18]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[19]  David Burshtein,et al.  Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes , 2000, IEEE Trans. Inf. Theory.

[20]  Shlomo Shamai,et al.  Variations on the Gallager bounds, connections, and applications , 2002, IEEE Trans. Inf. Theory.

[21]  D. Burshtein,et al.  Upper bounds on the rate of LDPC codes , 2002, Proceedings IEEE International Symposium on Information Theory,.

[22]  T. Richrdson,et al.  Finite-length analysis of various low-density parity-check ensembles for the binary erasure channel , 2002, Proceedings IEEE International Symposium on Information Theory,.

[23]  Alexander Barg,et al.  Error exponents of expander codes , 2002, IEEE Trans. Inf. Theory.

[24]  Amin Shokrollahi,et al.  Capacity-achieving sequences for the erasure channel , 2002, IEEE Trans. Inf. Theory.

[25]  Emre Telatar,et al.  Finite-length analysis of low-density parity-check codes on the binary erasure channel , 2002, IEEE Trans. Inf. Theory.

[26]  Aamod Dinkar Khandekar.  Graph-based codes and iterative decoding , 2003 .

[27]  Meir Feder,et al.  Bounds on achievable rates of LDPC codes used over the binary erasure channel , 2004, IEEE Transactions on Information Theory.