Zero-Delay and Causal Single-User and Multi-User Lossy Source Coding with Decoder Side Information

We consider zero-delay, single-user, and multi-user source coding with an average distortion constraint and decoder side information. The zero-delay constraint translates into causal (sequential) encoder and decoder pairs as well as the use of instantaneous codes. For the single-user setting, we show that optimal performance is attained by time sharing at most two scalar encoder-decoder pairs, that use zero-error side information codes. Side information look-ahead is shown to be useless in this setting. Furthermore, we show that even without delay constraints, if either the encoder or decoder are restricted a priori to be scalar, the performance loss cannot be compensated by the other component, which can be scalar as well without further loss. Finally, we show that the multi-terminal source coding problem can be solved in the zero-delay regime and the rate-distortion region is provided.

[1]  Noga Alon,et al.  Source coding and graph entropies , 1996, IEEE Trans. Inf. Theory.

[2]  H. S. WITSENHAUSEN,et al.  The zero-error side information problem and chromatic numbers (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[3]  Tsachy Weissman,et al.  On causal source codes with side information , 2005, IEEE Transactions on Information Theory.

[4]  Neri Merhav,et al.  Structure Theorems for Real-Time Variable Rate Coding With and Without Side Information , 2011, IEEE Transactions on Information Theory.

[5]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[6]  Kenneth Rose,et al.  Zero-error source coding with maximum distortion criterion , 2002, Proceedings DCC 2002. Data Compression Conference.

[7]  Alon Orlitsky,et al.  Average-case interactive communication , 1992, IEEE Trans. Inf. Theory.

[8]  Pramod Viswanath,et al.  Rate Region of the Quadratic Gaussian Two-Encoder Source-Coding Problem , 2008, IEEE Transactions on Information Theory.

[9]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[10]  Tamer Basar,et al.  Causal coding of markov sources with continuous alphabets , 2008 .

[11]  Tsachy Weissman,et al.  Source Coding with Limited Side Information Lookahead at the Decoder , 2006, 2006 IEEE International Symposium on Information Theory.

[12]  Neri Merhav,et al.  On the Statistical Physics of Directed Polymers in a Random Medium and Their Relation to Tree Codes , 2008, IEEE Transactions on Information Theory.

[13]  Toby Berger,et al.  Multiterminal source encoding with one distortion criterion , 1989, IEEE Trans. Inf. Theory.

[14]  Tsachy Weissman,et al.  Multiterminal Source Coding Under Logarithmic Loss , 2011, IEEE Transactions on Information Theory.

[15]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[16]  Neri Merhav,et al.  Data-Processing Bounds for Scalar Lossy Source Codes With Side Information at the Decoder , 2013, IEEE Transactions on Information Theory.

[17]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[18]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[19]  Aaron D. Wyner,et al.  On source coding with side information at the decoder , 1975, IEEE Trans. Inf. Theory.

[20]  N. THOMAS GAARDER,et al.  On optimal finite-state digital transmission systems , 1982, IEEE Trans. Inf. Theory.

[21]  Pramod Viswanath,et al.  Rate Region of the Quadratic Gaussian Two-Encoder Source-Coding Problem , 2006, ISIT.

[22]  Kenneth Rose,et al.  On zero-error source coding with decoder side information , 2003, IEEE Trans. Inf. Theory.

[23]  Michelle Effros,et al.  Quantization as Histogram Segmentation: Optimal Scalar Quantizer Design in Network Systems , 2008, IEEE Transactions on Information Theory.

[24]  Tsachy Weissman,et al.  Source Coding With Limited-Look-Ahead Side Information at the Decoder , 2006, IEEE Transactions on Information Theory.

[25]  David L. Neuhoff,et al.  Causal source codes , 1982, IEEE Trans. Inf. Theory.