Duality between channel capacity and rate distortion with two-sided state information

We show that the duality between channel capacity and data compression is retained when state information is available to the sender, to the receiver, to both, or to neither. We present a unified theory for eight special cases of channel capacity and rate distortion with state information, which also extends existing results to arbitrary pairs of independent and identically distributed (i.i.d.) correlated state information (S/sub 1/, S/sub 2/) available at the sender and at the receiver, respectively. In particular, the resulting general formula for channel capacity C = max/sub p/(u,x|s/sub 1/) [I(U; S/sub 2/, Y) I(U; S/sub 1/)] assumes the same form as the generalized Wyner-Ziv (1976) rate distortion function R(D) = min/sub p/(u|x, s/sub 1/)p(x/spl I.cap/|u, s/sub 2/) [I(U; S/sub 1/, X) 1(U; S/sub 2/)].

[1]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[2]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[3]  Robert J. McEliece,et al.  The Theory of Information and Coding , 1979 .

[4]  Abbas El Gamal,et al.  On the capacity of computer memory with defects , 1983, IEEE Trans. Inf. Theory.

[5]  Max H. M. Costa,et al.  Writing on dirty paper , 1983, IEEE Trans. Inf. Theory.

[6]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[7]  Aaron D. Wyner,et al.  Channels with Side Information at the Transmitter , 1993 .

[8]  Ram Zamir,et al.  The rate loss in the Wyner-Ziv problem , 1996, IEEE Trans. Inf. Theory.

[9]  S. Shamai,et al.  Nested linear/lattice codes for Wyner-Ziv encoding , 1998, 1998 Information Theory Workshop (Cat. No.98EX131).

[10]  Shlomo Shamai,et al.  On the capacity of some channels with channel state information , 1999, IEEE Trans. Inf. Theory.

[11]  J. Chou,et al.  On the duality between distributed source coding and data hiding , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[12]  Tamás Linder,et al.  On source coding with side-information-dependent distortion measures , 2000, IEEE Trans. Inf. Theory.

[13]  Tamás Linder,et al.  Optimal entropy-constrained scalar quantization of a uniform source , 2000, IEEE Trans. Inf. Theory.

[14]  Mung Chiang,et al.  Tradeoff between message and state information rates , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[15]  Mung Chiang,et al.  Unified duality between channel capacity and rate distortion with state information , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[16]  Gregory W. Wornell,et al.  The duality between information embedding and source coding with side information and some applications , 2003, IEEE Trans. Inf. Theory.