The Quadratic Gaussian Rate-Distortion Function for Source Uncorrelated Distortions

We characterize the rate-distortion function for zero-mean stationary Gaussian sources under the MSE fidelity criterion and subject to the additional constraint that the distortion is uncorrelated to the input. The solution is given by two equations coupled through a single scalar parameter. This has a structure similar to the well known water-filling solution obtained without the uncorrelated distortion restriction. Our results fully characterize the unique statistics of the optimal distortion. We also show that, for all positive distortions, the minimum achievable rate subject to the uncorrelation constraint is strictly larger than that given by the un-constrained rate-distortion function. This gap increases with the distortion and tends to infinity and zero, respectively, as the distortion tends to zero and infinity.

[1]  K. H. Barratt Digital Coding of Waveforms , 1985 .

[2]  R. Schreier,et al.  Delta-sigma data converters : theory, design, and simulation , 1997 .

[3]  John G. Proakis,et al.  Digital signal processing (3rd ed.): principles, algorithms, and applications , 1996 .

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[6]  Robert M. Gray,et al.  Dithered quantizers , 1993, IEEE Trans. Inf. Theory.

[7]  Yuval Kochman,et al.  Achieving the Gaussian Rate–Distortion Function by Prediction , 2007, IEEE Transactions on Information Theory.

[8]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Peter No,et al.  Digital Coding of Waveforms , 1986 .

[11]  N. Wiener,et al.  Fourier Transforms in the Complex Domain , 1934 .

[12]  Meir Feder,et al.  On universal quantization by randomized uniform/lattice quantizers , 1992, IEEE Trans. Inf. Theory.

[13]  Graham C. Goodwin,et al.  Fundamental Limitations in Filtering and Control , 1997 .

[14]  Vivek K. Goyal,et al.  Transform coding with backward adaptive updates , 2000, IEEE Trans. Inf. Theory.

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  R. Gallager Information Theory and Reliable Communication , 1968 .

[17]  A. Habibi,et al.  A Unified Representation of Differential Pulse-Code Modulation (DPCM) and Transform Coding Systems , 1974, IEEE Trans. Commun..

[18]  Suhas N. Diggavi,et al.  The worst additive noise under a covariance constraint , 2001, IEEE Trans. Inf. Theory.

[19]  See-May Phoong,et al.  Prediction-based lower triangular transform , 2000, IEEE Trans. Signal Process..

[20]  Vivek K. Goyal,et al.  Theoretical foundations of transform coding , 2001, IEEE Signal Process. Mag..

[21]  Meir Feder,et al.  On lattice quantization noise , 1996, IEEE Trans. Inf. Theory.

[22]  R. Gray,et al.  Dithered Quantizers , 1993, Proceedings. 1991 IEEE International Symposium on Information Theory.

[23]  Graham C. Goodwin,et al.  On Optimal Perfect Reconstruction Feedback Quantizers , 2008, IEEE Transactions on Signal Processing.

[24]  Graham C. Goodwin,et al.  Conditions for optimality of scalar feedback quantization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[25]  David L. Neuhoff,et al.  The validity of the additive noise model for uniform scalar quantizers , 2005, IEEE Transactions on Information Theory.

[26]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[27]  P. Schultheiss,et al.  Block Quantization of Correlated Gaussian Random Variables , 1963 .

[28]  P. Vaidyanathan Multirate Systems And Filter Banks , 1992 .