Vector quantization and density estimation

The connection between compression and the estimation of probability distributions has long been known for the case of discrete alphabet sources and lossless coding. A universal lossless code which does a good job of compressing must implicitly also do a good job of modeling. In particular, with a collection of codebooks, one for each possible class or model, if codewords are chosen from among the ensemble of codebooks so as to minimize bit rate, then the codebook selected provides an implicit estimate of the underlying class. Less is known about the corresponding connections between lossy compression and continuous sources. We consider aspects of estimating conditional and unconditional densities in conjunction with Bayes-risk weighted vector quantization for joint compression and classification.

[1]  Robert M. Gray,et al.  Bayes risk weighted tree-structured vector quantization with posterior estimation , 1994, Proceedings of IEEE Data Compression Conference (DCC'94).

[2]  Robert M. Gray,et al.  Asymptotic Performance of Vector Quantizers with a Perceptual Distortion Measure , 1997, IEEE Trans. Inf. Theory.

[3]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[4]  Robert M. Gray,et al.  Joint image classification and compression using hierarchical table-lookup vector quantization , 1996, Proceedings of Data Compression Conference - DCC '96.

[5]  R. Gray,et al.  Combining tree-structured vector quantization with classification and regression trees , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[6]  M. C. Jones,et al.  A Remark on Algorithm as 176. Kernel Density Estimation Using the Fast Fourier Transform , 1984 .

[7]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[8]  Saburo Tazaki,et al.  Asymptotic performance of block quantizers with difference distortion measures , 1980, IEEE Trans. Inf. Theory.

[9]  Robert M. Gray,et al.  Bayes risk weighted vector quantization with CART estimated class posteriors , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[10]  Bhaskar D. Rao,et al.  Theoretical analysis of the high-rate vector quantization of LPC parameters , 1995, IEEE Trans. Speech Audio Process..

[11]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[12]  Robert M. Gray,et al.  A comparison of Bayes risk weighted vector quantization with posterior estimation with other VQ-based classifiers , 1994, Proceedings of 1st International Conference on Image Processing.

[13]  Michelle Effros,et al.  A vector quantization approach to universal noiseless coding and quantization , 1996, IEEE Trans. Inf. Theory.

[14]  Robert M. Gray,et al.  Text segmentation in mixed-mode images using classification trees and transform tree-structured vector quantization , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[15]  G. Lugosi,et al.  Consistency of Data-driven Histogram Methods for Density Estimation and Classification , 1996 .

[16]  Wayne E. Stark,et al.  Fine-coarse vector quantization , 1991, IEEE Trans. Signal Process..

[17]  Robert M. Gray,et al.  Evaluation of Bayes risk weighted vector quantization with posterior estimation in the detection of lesions in digitized mammograms , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[18]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[19]  Robert M. Gray,et al.  Bayes risk weighted vector quantization with posterior estimation for image compression and classification , 1996, IEEE Trans. Image Process..

[20]  Pamela C. Cosman,et al.  Incorporating visual factors into vector quantizers for image compression , 1993 .

[21]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[22]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[23]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[24]  Brian Bouzas,et al.  Objective image quality measure derived from digital image power spectra , 1992 .

[25]  Bhaskar Ramamurthi,et al.  Image coding using vector quantization , 1982, ICASSP.

[26]  Pao-Chi Chang,et al.  Hierarchical vector quantizers with table-lookup encoders , 1985 .

[27]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[28]  K. Oehler Image compression and classification using vector quantization , 1993 .

[29]  Robert M. Gray,et al.  Multiple local optima in vector quantizers , 1982, IEEE Trans. Inf. Theory.

[30]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[31]  Robert M. Gray,et al.  Rate-distortion speech coding with a minimum discrimination information distortion measure , 1981, IEEE Trans. Inf. Theory.

[32]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[33]  Philip A. Chou,et al.  Optimal pruning with applications to tree-structured source coding and modeling , 1989, IEEE Trans. Inf. Theory.

[34]  Richard D. Wesel,et al.  Bayes risk weighted VQ and learning VQ , 1994, Proceedings of IEEE Data Compression Conference (DCC'94).

[35]  David L. Neuhoff,et al.  Bennett's integral for vector quantizers , 1995, IEEE Trans. Inf. Theory.

[36]  Allen Gersho,et al.  Asymptotically optimal block quantization , 1979, IEEE Trans. Inf. Theory.

[37]  David L. Neuhoff,et al.  Theory of lattice-based fine-coarse vector quantization , 1991, IEEE Trans. Inf. Theory.

[38]  T. Kohonen,et al.  Statistical pattern recognition with neural networks: benchmarking studies , 1988, IEEE 1988 International Conference on Neural Networks.

[39]  B. Silverman,et al.  Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[40]  B. Silverman,et al.  Algorithm AS 176: Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[41]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[42]  R. Gray,et al.  Combining Image Compression and Classification Using Vector Quantization , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Robert M. Gray,et al.  Combining image classification and image compression using vector quantization , 1993, [Proceedings] DCC `93: Data Compression Conference.

[44]  Paul L. Zador,et al.  Asymptotic quantization error of continuous signals and the quantization dimension , 1982, IEEE Trans. Inf. Theory.

[45]  W. R. Bennett,et al.  Spectra of quantized signals , 1948, Bell Syst. Tech. J..

[46]  P. Zador DEVELOPMENT AND EVALUATION OF PROCEDURES FOR QUANTIZING MULTIVARIATE DISTRIBUTIONS , 1963 .