Source coding with delayed side information

For memoryless sources, delayed side information at the decoder does not improve the rate-distortion function. However, this is not the case for more general sources with memory, as demonstrated by a number of works focusing on the special case of (delayed) feedforward. In this paper, a setting is studied in which the side information is delayed and the encoder is informed about the side information sequence. Assuming a hidden Markov model for the sources, at first, a single-letter characterization is given for the set-up where the side information delay is arbitrary and known at the encoder, and the reconstruction at the destination is required to be (near) lossless. Then, with delay equal to zero or one source symbol, a single-letter characterization is given of the rate-distortion function for the case where side information may be delayed or not, unbeknownst to the encoder. Finally, an example for a binary source is provided.

[1]  Robert M. Gray,et al.  Information rates of autoregressive processes , 1970, IEEE Trans. Inf. Theory.

[2]  Ramji Venkataramanan,et al.  Achievable Rates for Multiple Descriptions With Feed-Forward , 2011, IEEE Transactions on Information Theory.

[3]  Tsachy Weissman,et al.  Source Coding With Limited-Look-Ahead Side Information at the Decoder , 2006, IEEE Transactions on Information Theory.

[4]  Tsachy Weissman,et al.  On competitive prediction and its relation to rate-distortion theory , 2003, IEEE Trans. Inf. Theory.

[5]  Ramji Venkataramanan,et al.  Directed information for communication problems with common side information and delayed feedback/feedforward , 2005 .

[6]  Haim H. Permuter,et al.  Source Coding When the Side Information May Be Delayed , 2011, IEEE Transactions on Information Theory.

[7]  Ramji Venkataramanan,et al.  On Computing the Feedback Capacity of Channels and the Feed-Forward Rate-Distortion Function of Sources , 2010, IEEE Transactions on Communications.

[8]  Ramji Venkataramanan,et al.  Source Coding With Feed-Forward: Rate-Distortion Theorems and Error Exponents for a General Source , 2007, IEEE Transactions on Information Theory.

[9]  Haim H. Permuter,et al.  Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources , 2013, IEEE Transactions on Information Theory.

[10]  Amiram H. Kaspi,et al.  Rate-distortion function when side-information may be present at the decoder , 1994, IEEE Trans. Inf. Theory.