Efficient On-Line Schemes for Encoding Individual Sequences With Side Information at the Decoder

We present adaptive on-line schemes for lossy encoding of individual sequences, under the conditions of the Wyner-Ziv (WZ) problem. In the first part of this paper, a set of fixed-rate scalar source codes with zero delay is presented. We propose a randomized on-line coding scheme, which achieves asymptotically (and with high probability), the performance of the best source code in the set, uniformly over all source sequences. Efficient algorithms for implementing this scheme for small and large sets of encoders are presented. In the second part of this work, we generalize our results to the case of variable-rate coding. A set of variable-rate scalar source codes is presented. This time, the performance is measured by the Lagrangian Cost (LC), which is defined as a weighted sum of the distortion and the length of the encoded sequence. Efficient algorithms for implementing the generalized on-line coding scheme are presented. We then consider the special case of lossless variable-rate coding. An on-line scheme which uses Huffman codes is presented. We show that this scheme can be implemented efficiently using the same graphic methods from the first part. Finally, combining the results from former sections, we build a generalized efficient algorithm for a structured set of variable-rate encoders.

[1]  Demosthenis Teneketzis,et al.  On the Structure of Optimal Real-Time Encoders and Decoders in Noisy Communication , 2006, IEEE Transactions on Information Theory.

[2]  Tsachy Weissman,et al.  Universal Zero-Delay Joint Source–Channel Coding , 2006, IEEE Transactions on Information Theory.

[3]  Tamás Linder,et al.  A "follow the perturbed leader"-type algorithm for zero-delay quantization of individual sequences , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[4]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[5]  Jeffrey Scott Vitter,et al.  Design and analysis of dynamic Huffman codes , 1987, JACM.

[6]  Tsachy Weissman,et al.  On limited-delay lossy coding and filtering of individual sequences , 2002, IEEE Trans. Inf. Theory.

[7]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[8]  Tamás Linder,et al.  Tracking the Best Quantizer , 2005, IEEE Transactions on Information Theory.

[9]  Tamás Linder,et al.  On the structure of optimal entropy-constrained scalar quantizers , 2002, IEEE Trans. Inf. Theory.

[10]  Neri Merhav,et al.  On the Wyner-Ziv problem for individual sequences , 2006, IEEE Transactions on Information Theory.

[11]  Kannan Ramchandran,et al.  Distributed compression for sensor networks , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[12]  Tamás Linder,et al.  Efficient adaptive algorithms and minimax bounds for zero-delay lossy source coding , 2004, IEEE Transactions on Signal Processing.

[13]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[14]  Michelle Effros,et al.  Codecell contiguity in optimal fixed-rate and entropy-constrained network scalar quantizers , 2002, Proceedings DCC 2002. Data Compression Conference.

[15]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[16]  Neri Merhav,et al.  Efficient on-line schemes for encoding individual sequences with side information at the decoder , 2009, ISIT.

[17]  Jacob Ziv.  Fixed-rate encoding of individual sequences with side information , 1984, IEEE Trans. Inf. Theory.

[18]  Manfred K. Warmuth,et al.  Path Kernels and Multiplicative Updates , 2002, J. Mach. Learn. Res..

[19]  Tamás Linder,et al.  A zero-delay sequential scheme for lossy coding of individual sequences , 2001, IEEE Trans. Inf. Theory.

[20]  Tsachy Weissman,et al.  A Universal Wyner-Ziv Scheme for Discrete Sources , 2007, 2007 IEEE International Symposium on Information Theory.

[21]  Michelle Effros,et al.  Quantization as Histogram Segmentation: Optimal Scalar Quantizer Design in Network Systems , 2008, IEEE Transactions on Information Theory.

[22]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .