Successive refinement for the Wyner-Ziv problem and layered code design

We examine successive refinement for the Wyner-Ziv problem described in a recent paper by Steinberg and Merhav, where the authors showed that if the side information for all stages is identical, then the jointly Gaussian source with squared error distortion measure is successively refinable. We first extend this result to the case where the difference between the source and the side information is Gaussian and independent of the side information. As a byproduct, we give an alternative proof that the Wyner-Ziv problem for these sources has no rate loss-a result that was recently shown by Pradhan et al. through invoking the duality between the Gaussian Wyner-Ziv problem and the Costa problem. We then perform layered Wyner-Ziv code design for this general type of source based on nested scalar quantization, bit-plane coding, and low-density parity check (LDPC) code-based Slepian-Wolf coding (source coding with side information). We show that density evolution can be used to analyze the Slepian-Wolf code performance, provided that certain symmetry conditions, which have been dubbed dual symmetry, are satisfied by the hypothetical channel between the source and the side information. We also show that the dual symmetry condition is indeed satisfied by the hypothetical channel in our Slepian-Wolf coding setup. This justifies the use of density evolution in our LDPC code-based Slepian-Wolf code design for Wyner-Ziv coding. For the jointly Gaussian source, our layered coder performs 1.29 to 3.45 dB from the Wyner-Ziv bound for rates ranging from 0.47 to 4.68 bits per sample. When the side information is Laplacian and the source equals the side information plus an independent Gaussian noise term, our layered coder performs 1.33 to 3.90 dB from the Wyner-Ziv bound for rates ranging from 0.48 to 4.64 bits per sample.

[1]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[2]  Rüdiger L. Urbanke,et al.  Design of capacity-approaching irregular low-density parity-check codes , 2001, IEEE Trans. Inf. Theory.

[3]  William Equitz,et al.  Successive refinement of information , 1991, IEEE Trans. Inf. Theory.

[4]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[7]  Sae-Young Chung,et al.  Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation , 2001, IEEE Trans. Inf. Theory.

[8]  Max H. M. Costa,et al.  Writing on dirty paper , 1983, IEEE Trans. Inf. Theory.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Zixiang Xiong,et al.  Channel symmetry in Slepian-Wolf code design based on LDPC codes with application to the quadratic Gaussian Wyner-Ziv problem , 2004, Information Theory Workshop.

[11]  B. Girod,et al.  Transforms for high-rate distributed source coding , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[12]  Zixiang Xiong,et al.  Layered Wyner-Ziv video coding , 2004, IS&T/SPIE Electronic Imaging.

[13]  Neri Merhav,et al.  On successive refinement for the Wyner-Ziv problem , 2004, IEEE Transactions on Information Theory.

[14]  Amir H. Banihashemi,et al.  A heuristic search for good low-density parity-check codes at short block lengths , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[15]  Zixiang Xiong,et al.  Successive refinement for the Wyner-Ziv problem and layered code design , 2004, IEEE Transactions on Signal Processing.

[16]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[17]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[18]  Zixiang Xiong,et al.  Compression of binary sources with side information at the decoder using LDPC codes , 2002, IEEE Communications Letters.

[19]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[20]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[21]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[22]  Shlomo Shamai,et al.  Nested linear/Lattice codes for structured multiterminal binning , 2002, IEEE Trans. Inf. Theory.

[23]  David J. C. MacKay,et al.  Comparison of constructions of irregular Gallager codes , 1999, IEEE Trans. Commun..

[24]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[25]  Kannan Ramchandran,et al.  Duality between source coding and channel coding and its extension to the side information case , 2003, IEEE Trans. Inf. Theory.

[26]  Zixiang Xiong,et al.  Slepian-Wolf coded nested quantization (SWC-NQ) for Wyner-Ziv coding: performance analysis and code design , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.