On the structure of optimal entropy-constrained scalar quantizers

The nearest neighbor condition implies that when searching for a mean-square optimal fixed-rate quantizer it is enough to consider the class of regular quantizers, i.e., quantizers having convex cells and codepoints which lie inside the associated cells. In contrast, quantizer regularity can preclude optimality in entropy-constrained quantization. This can be seen by exhibiting a simple discrete scalar source for which the mean-square optimal entropy-constrained scalar quantizer (ECSQ) has disconnected (and hence nonconvex) cells at certain rates. In this work, new results concerning the structure and existence of optimal ECSQs are presented. One main result shows that for continuous sources and distortion measures of the form d(x,y)=/spl rho/(|x-y|), where /spl rho/ is a nondecreasing convex function, any finite-level ECSQ can be "regularized" so that the resulting regular quantizer has the same entropy and equal or less distortion. Regarding the existence of optimal ECSQs, we prove that under rather general conditions there exists an "almost regular" optimal ECSQ for any entropy constraint. For the squared error distortion measure and sources with piecewise-monotone and continuous densities, the existence of a regular optimal ECSQ is shown.

[1]  Herbert Gish,et al.  Asymptotically efficient quantizing , 1968, IEEE Trans. Inf. Theory.

[2]  Paul L. Zador,et al.  Asymptotic quantization error of continuous signals and the quantization dimension , 1982, IEEE Trans. Inf. Theory.

[3]  Joel Max,et al.  Quantizing for minimum distortion , 1960, IRE Trans. Inf. Theory.

[4]  F. Jones.  Lebesgue Integration on Euclidean Space , 1993 .

[5]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[6]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[7]  Robert M. Gray,et al.  Locally Optimal Block Quantizer Design , 1980, Inf. Control..

[8]  Nariman Farvardin,et al.  Optimum quantizer performance for a class of non-Gaussian memoryless sources , 1984, IEEE Trans. Inf. Theory.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[11]  R. Ladner.  Entropy-constrained Vector Quantization , 2000 .

[12]  Ram Zamir,et al.  Causal source coding of stationary sources with high resolution , 2001, Proceedings. 2001 IEEE International Symposium on Information Theory (IEEE Cat. No.01CH37252).

[13]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[14]  Robert M. Gray,et al.  Quantizer Mismatch , 1975, IEEE Trans. Commun..

[15]  David Pollard,et al.  Quantization and the method of k -means , 1982, IEEE Trans. Inf. Theory.

[16]  Robert M. Gray,et al.  Global convergence and empirical consistency of the generalized Lloyd algorithm , 1986, IEEE Trans. Inf. Theory.

[17]  G. Wise,et al.  Convergence of Vector Quantizers with Applications to Optimal Quantization , 1984 .

[18]  Tamás Linder,et al.  Optimal entropy-constrained scalar quantization of a uniform source , 2000, IEEE Trans. Inf. Theory.

[19]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[20]  John C. Kieffer,et al.  New results on optimal entropy-constrained quantization , 1988, IEEE Trans. Inf. Theory.

[21]  A. N. Netravali,et al.  Optimum quantizer design using a fixed-point algorithm , 1976, The Bell System Technical Journal.

[22]  Toby Berger.  Optimum quantizers and permutation codes , 1972, IEEE Trans. Inf. Theory.

[23]  Philip A. Chou,et al.  Entropy-constrained vector quantization , 1989, IEEE Trans. Acoust. Speech Signal Process..

[24]  Aaron D. Wyner,et al.  An Upper Bound on the Entropy Series , 1972, Inf. Control..

[25]  S. Graf,et al.  Foundations of Quantization for Probability Distributions , 2000 .

[26]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[27]  P. A. Chou,et al.  When optimal entropy-constrained quantizers have only a finite number of codewords , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).