Near-Lossless Compression for Sparse Source Using Convolutional Low Density Generator Matrix Codes

In this paper, we present a new coding approach to near-lossless compression for binary sparse sources by using a special class of low density generator matrix (LDGM) codes. On the theoretical side, we proved that such a class of block LDGM codes are universal in the sense that any source with an entropy less than the coding rate can be compressed and reconstructed with an arbitrarily low bit-error rate (BER). On the practical side, we employ spatially coupled LDGM codes to reduce the complexity of reconstruction by implementing an iterative sliding window decoding algorithm. Figure of merits of the proposed scheme include its flexibility and universality. The encoder does not require the knowledge of the source statistic, while the decoder can estimate easily the source parameter as required by the iterative decoding. The implementation complexity is analyzed and the performance is simulated. Numerical results show that the proposed scheme performs well over a wide range of sources.

[1]  S. Verdú,et al.  Noiseless Data Compression with Low-Density Parity-Check Codes , 2003, Advances in Network Information Theory.

[2]  Jörg Kliewer,et al.  Encoding of Spatially Coupled LDGM Codes for Lossy Source Compression , 2018, IEEE Transactions on Communications.

[3]  Pedram Pad,et al.  Capacity achieving linear codes with random binary sparse generating matrices over the Binary Symmetric Channel , 2011, 2012 IEEE International Symposium on Information Theory Proceedings.

[4]  Ying Zhao,et al.  Compression of binary memoryless sources using punctured turbo codes , 2002, IEEE Communications Letters.

[5]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[6]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[7]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[8]  Nicolas Macris,et al.  Approaching the rate-distortion limit by spatial coupling with belief propagation and decimation , 2013, 2013 IEEE International Symposium on Information Theory.

[9]  Wenchao Lin,et al.  Systematic Convolutional Low Density Generator Matrix Code , 2020, IEEE Transactions on Information Theory.

[10]  Martin J. Wainwright,et al.  Lossy Source Compression Using Low-Density Generator Matrix Codes: Analysis and Algorithms , 2010, IEEE Transactions on Information Theory.

[11]  Xiao Ma.  Coding theorem for systematic low density generator matrix codes , 2016, 2016 9th International Symposium on Turbo Codes and Iterative Information Processing (ISTC).

[12]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .