Low-density constructions can achieve the Wyner-Ziv and Gelfand-Pinsker bounds

We describe and analyze sparse graphical code constructions for the problems of source coding with decoder side information (the Wyner-Ziv problem), and channel coding with encoder side information (the Gelfand-Pinsker problem). Our approach relies on a combination of low-density parity check (LDPC) codes and low-density generator matrix (LDGM) codes, and produces sparse constructions that are simultaneously good as both source and channel codes. In particular, we prove that under maximum likelihood encoding/decoding, there exist low-density codes (i.e., with finite degrees) from our constructions that can saturate both the Wyner-Ziv and Gelfand-Pinsker bounds

[1]  Andrew J. Viterbi,et al.  Trellis Encoding of memoryless discrete-time sources with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[2]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[3]  J. Wolfowitz.  The rate distortion function for source coding with side information at the decoder , 1979 .

[4]  Michael W. Marcellin,et al.  Trellis coded quantization of memoryless and Gauss-Markov sources , 1990, IEEE Trans. Commun..

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[7]  Shlomo Shamai,et al.  Nested linear/Lattice codes for structured multiterminal binning , 2002, IEEE Trans. Inf. Theory.

[8]  Simon Litsyn,et al.  On ensembles of low-density parity-check codes: Asymptotic distance distributions , 2002, IEEE Trans. Inf. Theory.

[9]  Kannan Ramchandran,et al.  Turbo and trellis-based constructions for source coding with side information , 2003, Data Compression Conference, 2003. Proceedings. DCC 2003.

[10]  Gregory W. Wornell,et al.  The duality between information embedding and source coding with side information and some applications , 2003, IEEE Trans. Inf. Theory.

[11]  T. Murayama.  Thouless-Anderson-Palmer approach for lossy compression. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Emin Martinian,et al.  Iterative Quantization Using Codes On Graphs , 2004, ArXiv.

[13]  Stephan ten Brink,et al.  A close-to-capacity dirty paper coding scheme , 2004, IEEE Transactions on Information Theory.

[14]  Martin J. Wainwright,et al.  Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[15]  Zixiang Xiong,et al.  On Multiterminal Source Code Design , 2005, IEEE Transactions on Information Theory.

[16]  R. Zecchina,et al.  Message-Passing Algorithms for Non-Linear Nodes and Data Compression , 2005, Complexus.

[17]  Martin J. Wainwright,et al.  Low density codes achieve the rate-distortion bound , 2006, Data Compression Conference (DCC'06).