Achieving the Gaussian Rate-Distortion Function by Prediction

The "water-filling" solution for the quadratic rate-distortion distortion function of a stationary Gaussian source is given in terms of its power spectrum. This formula naturally lends itself to a frequency domain "test-channel" realization. We provide an alternative time-domain realization for the rate-distortion function, based on linear prediction. This solution has some interesting implications, including the optimality at all distortion levels of pre/post filtered vector-quantized differential pulse code modulation (DPCM), and a duality relationship with decision-feedback equalization (DFE) for inter-symbol interference (ISI) channels.

[1]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[2]  Mahesh K. Varanasi,et al.  An information-theoretic framework for deriving canonical decision-feedback receivers in Gaussian channels , 2005, IEEE Transactions on Information Theory.

[3]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[4]  Tamás Linder,et al.  Causal coding of stationary sources and individual sequences with high resolution , 2006, IEEE Transactions on Information Theory.

[5]  Peter No,et al.  Digital Coding of Waveforms , 1986 .

[6]  Meir Feder,et al.  On universal quantization by randomized uniform/lattice quantizers , 1992, IEEE Trans. Inf. Theory.

[7]  Meir Feder,et al.  On lattice quantization noise , 1996, IEEE Trans. Inf. Theory.

[8]  Uri Erez,et al.  Achieving 1/2 log (1+SNR) on the AWGN channel with lattice encoding and decoding , 2004, IEEE Transactions on Information Theory.

[9]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[10]  Meir Feder,et al.  Information rates of pre/post-filtered dithered quantizers , 1993, IEEE Trans. Inf. Theory.

[11]  J. Massey.  CAUSALITY, FEEDBACK AND DIRECTED INFORMATION , 1990 .

[12]  John M. Cioffi,et al.  MMSE decision-feedback equalizers and coding. I. Equalization results , 1995, IEEE Trans. Commun..

[13]  Toby Berger,et al.  Digital Compression For Multimedia Principles And Standards , 1998 .

[14]  Chao Tian,et al.  Multiple Description Quantization Via Gram–Schmidt Orthogonalization , 2005, IEEE Transactions on Information Theory.

[15]  Toby Berger,et al.  Sending a lossy version of the innovations process is suboptimal in QG rate-distortion , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[16]  Michael T. Orchard,et al.  On the DPCM compression of Gaussian autoregressive sequences , 2001, IEEE Trans. Inf. Theory.

[17]  G. David Forney,et al.  Shannon meets Wiener II: On MMSE estimation in successive decoding schemes , 2004, ArXiv.

[18]  John G. Proakis,et al.  Digital Communications , 1983 .