A Unified Perspective on Parity- and Syndrome-Based Binary Data Compression Using Off-the-Shelf Turbo Codecs

We consider the problem of compressing memoryless binary data with or without side information at the decoder. We review the parity- and the syndrome-based approaches and discuss their theoretical limits, assuming that there exists a virtual binary symmetric channel between the source and the side information, and that the source is not necessarily uniformly distributed. We take a factor-graph-based approach in order to devise how to take full advantage of the ready-available iterative decoding procedures when turbo codes are employed, in both a parity- or a syndrome-based fashion. We end up obtaining a unified decoder formulation that holds both for error-free and for error-prone encoder-to-decoder transmission over generic channels. To support the theoretical results, the different compression systems analyzed in the paper are also experimentally tested. They are compared against several different approaches proposed in literature and shown to be competitive in a variety of cases.

[1]  Hend Alqamzi,et al.  An optimal distributed and adaptive source coding strategy using rate-compatible punctured convolutional codes , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[2]  Jing Li,et al.  Enhancing the robustness of distributed compression using ideas from channel coding , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[3]  Rick S. Blum,et al.  Slepian-Wolf coding for nonuniform sources using turbo codes , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[4]  Ying Zhao,et al.  Compression of binary memoryless sources using punctured turbo codes , 2002, IEEE Communications Letters.

[5]  G. David Forney,et al.  Coset codes-I: Introduction and geometrical classification , 1988, IEEE Trans. Inf. Theory.

[6]  Zixiang Xiong,et al.  Distributed source coding for sensor networks , 2004, IEEE Signal Processing Magazine.

[7]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[8]  Ying Zhao,et al.  Compression of correlated binary sources using turbo codes , 2001, IEEE Communications Letters.

[9]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[10]  Joachim Hagenauer,et al.  LOSSLESS TURBO SOURCE CODING WITH DECREMENTAL REDUNDANCY , 2004 .

[11]  Kannan Ramchandran,et al.  Distributed source coding using syndromes (DISCUSS): design and construction , 1999 .

[12]  Joachim Hagenauer,et al.  The ' hrbo Principle in Joint Source-Channel Coding , 2003 .

[13]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[14]  Rick S. Blum,et al.  An Efficient SF-ISF Approach for the Slepian-Wolf Source Coding Problem , 2005, EURASIP J. Adv. Signal Process..

[15]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[16]  B. Girod,et al.  Rate-Adaptive Distributed Source Coding using Low-Density Parity-Check Codes , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[17]  Zixiang Xiong,et al.  Distributed compression of binary sources using conventional parallel and serial concatenated convolutional codes , 2003, Data Compression Conference, 2003. Proceedings. DCC 2003.

[18]  Zixiang Xiong,et al.  Joint source-channel coding of binary sources with side information at the decoder using IRA codes , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[19]  Kannan Ramchandran,et al.  Distributed compression in a dense microsensor network , 2002, IEEE Signal Process. Mag..

[20]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[21]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[22]  Norbert Goertz,et al.  The turbo principle in joint source-channel coding , 2003, Proceedings 2003 IEEE Information Theory Workshop (Cat. No.03EX674).

[23]  Jing Li,et al.  Slepian-Wolf Coding Using Parity Approach and Syndrome Approach , 2007, 2007 41st Annual Conference on Information Sciences and Systems.

[24]  S. Shamai,et al.  Nested linear/lattice codes for Wyner-Ziv encoding , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[25]  Mina Sartipi,et al.  Distributed source coding using short to moderate length rate-compatible LDPC codes: the entire Slepian-Wolf rate region , 2008, IEEE Transactions on Communications.

[26]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[27]  Shlomo Shamai,et al.  Nested linear/Lattice codes for structured multiterminal binning , 2002, IEEE Trans. Inf. Theory.

[28]  Farshad Lahouti,et al.  A flexible rate slepian-wolf code construction , 2009, IEEE Transactions on Communications.

[29]  Aaron D. Wyner,et al.  Recent results in the Shannon theory , 1974, IEEE Trans. Inf. Theory.

[30]  D. Costello,et al.  New classes of algebraic interleavers for turbo-codes , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[31]  Lorenzo Cappellari.  On syndrome decoding for Slepian-Wolf coding based on convolutional and turbo codes , 2010, IEEE Communications Letters.

[32]  Kannan Ramchandran,et al.  Distributed code constructions for the entire Slepian-Wolf rate region for arbitrarily correlated sources , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[33]  Kannan Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 2003, IEEE Trans. Inf. Theory.

[34]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[35]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[36]  Jing Li,et al.  A new coding scheme for the noisy-channel Slepian-Wolf problem: separate design and joint decoding , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[37]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[38]  Zixiang Xiong,et al.  Compression of binary sources with side information at the decoder using LDPC codes , 2002, IEEE Communications Letters.

[39]  Kannan Ramchandran,et al.  Generalized coset codes for distributed binning , 2005, IEEE Transactions on Information Theory.

[40]  Bernd Girod,et al.  Compression with side information using turbo codes , 2002, Proceedings DCC 2002. Data Compression Conference.

[41]  Zixiang Xiong,et al.  On code design for the Slepian-Wolf problem and lossless multiterminal networks , 2006, IEEE Transactions on Information Theory.

[42]  Vladimir Sidorenko,et al.  Decoding of convolutional codes using a syndrome trellis , 1994, IEEE Trans. Inf. Theory.

[43]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[44]  Bernd Girod,et al.  Distributed Video Coding , 2005, Proceedings of the IEEE.

[45]  C. Guillemot,et al.  Rate-adaptive turbo-syndrome scheme for Slepian-Wolf Coding , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.