Construction of Universal Codes Using LDPC Matrices and Their Error Exponents

A universal coding scheme for information from i.i.d., arbitrarily varying sources, or memoryless correlated sources is constructed using LDPC matrices and shown to have an exponential upper bound of decoding error probability. As a corollary, we construct a universal code for the noisy channel model, which is not necessarily BSC. Simulation results show universality of the code with sum-product decoding, and presence of a gap between the error exponent obtained by simulation and that obtained theoretically.

[1]  Imre Csiszár.  Linear codes for sources and source networks: Error exponents, universal coding , 1982, IEEE Trans. Inf. Theory.

[2]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[3]  Zixiang Xiong,et al.  Distributed source coding for sensor networks , 2004, IEEE Signal Processing Magazine.

[4]  Shigeki Miyake,et al.  Lossy Data Compression over Zq by LDPC Code , 2006, 2006 IEEE International Symposium on Information Theory.

[5]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[6]  S. Verdú,et al.  Noiseless Data Compression with Low-Density Parity-Check Codes , 2003, Advances in Network Information Theory.

[7]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[8]  Sanjeev R. Kulkarni,et al.  Universal variable-to-fixed length source codes , 2001, IEEE Trans. Inf. Theory.

[9]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[10]  S. Shamai,et al.  Lossless data compression with error correcting codes , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[11]  Jun Muramatsu,et al.  Low-density parity-check matrices for coding of correlated sources , 2003, IEEE Transactions on Information Theory.

[12]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[13]  Uri Erez,et al.  The ML decoding performance of LDPC ensembles over Z/sub q/ , 2005, IEEE Transactions on Information Theory.

[14]  Giuseppe Caire.  Universal data compression with LDPC codes , 2003 .

[15]  Hirosuke Yamamoto,et al.  A coding theorem for lossy data compression by LDPC codes , 2003, IEEE Trans. Inf. Theory.

[16]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[17]  David Burshtein,et al.  Bounds on the maximum-likelihood decoding error probability of low-density parity-check codes , 2000, IEEE Trans. Inf. Theory.

[18]  Meir Feder,et al.  Random coding techniques for nonrandom codes , 1999, IEEE Trans. Inf. Theory.

[19]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.