Bayes risk weighted vector quantization with posterior estimation for image compression and classification

Classification and compression play important roles in communicating digital information. Their combination is useful in many applications, including the detection of abnormalities in compressed medical images. In view of the similarities of compression and low-level classification, it is not surprising that there are many similar methods for their design. Because some of these methods are useful for designing vector quantizers, it seems natural that vector quantization (VQ) is explored for the combined goal. We investigate several VQ-based algorithms that seek to minimize both the distortion of compressed images and errors in classifying their pixel blocks. These algorithms are investigated with both full search and tree-structured codes. We emphasize a nonparametric technique that minimizes both error measures simultaneously by incorporating a Bayes risk component into the distortion measure used for the design and encoding. We introduce a tree-structured posterior estimator to produce the class posterior probabilities required for the Bayes risk computation in this design. For two different image sources, we demonstrate that this system provides superior classification while maintaining compression close or superior to that of several other VQ-based designs, including Kohonen's (1992) "learning vector quantizer" and a sequential quantizer/classifier design.

[1]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[2]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[3]  H. V. Poor,et al.  Applications of Ali-Silvey Distance Measures in the Design of Generalized Quantizers for Binary Decision Systems , 1977, IEEE Trans. Commun..

[4]  E. E. Hilbert.  Cluster compression algorithm: A joint clustering/data compression concept , 1977 .

[5]  R. Olshen,et al.  Asymptotically Efficient Solutions to the Classification Problem , 1978 .

[6]  R. Olshen,et al.  Consistent nonparametric regression from recursive partitioning schemes , 1980 .

[7]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  Robert M. Gray,et al.  Minimum Cross-Entropy Pattern Classification and Cluster Analysis , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  R. Olshen,et al.  Almost surely consistent nonparametric regression from recursive partitioning schemes , 1984 .

[11]  Bhaskar Ramamurthi,et al.  Classified Vector Quantization of Images , 1986, IEEE Trans. Commun..

[12]  Robert M. Gray,et al.  Global convergence and empirical consistency of the generalized Lloyd algorithm , 1986, IEEE Trans. Inf. Theory.

[13]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[14]  T. Kohonen,et al.  Statistical pattern recognition with neural networks: benchmarking studies , 1988, IEEE 1988 International Conference on Neural Networks.

[15]  Teuvo Kohonen,et al.  An introduction to neural computing , 1988, Neural Networks.

[16]  R. Gray,et al.  Applications of information theory to pattern recognition and the design of decision trees and trellises , 1988 .

[17]  Philip A. Chou,et al.  Optimal pruning with applications to tree-structured source coding and modeling , 1989, IEEE Trans. Inf. Theory.

[18]  H. Abut,et al.  Vector Quantization , 1990 .

[19]  R. Gray,et al.  Variable rate vector quantization of images , 1990 .

[20]  R.M. Gray,et al.  A greedy tree growing algorithm for the design of variable rate vector quantizers [image compression] , 1991, IEEE Trans. Signal Process..

[21]  R. Gray,et al.  Classification using vector quantization , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[22]  R. Olshen,et al.  Predicting chemically induced duodenal ulcer and adrenal necrosis with classification trees. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[24]  Gerard F. McLean.  Vector quantization for texture classification , 1993, IEEE Trans. Syst. Man Cybern..

[25]  R. Gray,et al.  Combining tree-structured vector quantization with classification and regression trees , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[26]  Pamela C. Cosman,et al.  Tree-structured vector quantization of CT chest scans: image quality and diagnostic accuracy , 1993, IEEE Trans. Medical Imaging.

[27]  K. Oehler.  Image compression and classification using vector quantization , 1993 .

[28]  Kris Popat,et al.  Novel cluster-based probability model for texture synthesis, classification, and compression , 1993, Other Conferences.

[29]  Rabab Kreidieh Ward,et al.  Vector Quantization Technique for Nonparametric Classifier Design , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Robert M. Gray,et al.  Combining image classification and image compression using vector quantization , 1993, [Proceedings] DCC `93: Data Compression Conference.

[31]  Brian D. Ripley,et al.  Neural Networks and Related Methods for Classification , 1994 .

[32]  Kris Popat,et al.  Cluster-based probability model applied to image restoration and compression , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Robert M. Gray,et al.  A comparison of Bayes risk weighted vector quantization with posterior estimation with other VQ-based classifiers , 1994, Proceedings of 1st International Conference on Image Processing.

[34]  Richard D. Wesel,et al.  Bayes risk weighted VQ and learning VQ , 1994, Proceedings of IEEE Data Compression Conference (DCC'94).

[35]  D. Lyons,et al.  Vector quantizer designs for joint compression and terrain categorization of multispectral imagery , 1994 .

[36]  Robert M. Gray,et al.  Bayes risk weighted vector quantization with CART estimated class posteriors , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[37]  R. Gray,et al.  Combining Image Compression and Classification Using Vector Quantization , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Andrew B. Nobel,et al.  Termination and continuity of greedy growing for tree-structured vector quantizers , 1996, IEEE Trans. Inf. Theory.

[39]  Jorma Laaksonen,et al.  LVQ_PAK: The Learning Vector Quantization Program Package , 1996 .

[40]  G. Lugosi,et al.  Consistency of Data-driven Histogram Methods for Density Estimation and Classification , 1996 .

[41]  A. Nobel.  Histogram regression estimation using data-dependent partitions , 1996 .

[42]  Andrew B. Nobel,et al.  Recursive partitioning to reduce distortion , 1997, IEEE Trans. Inf. Theory.