Optimal pruning with applications to tree-structured source coding and modeling

An algorithm introduced by L. Breiman et al. (1984) in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved. These include variable-rate and minimum-entropy tree-structured vector quantization, minimum expected cost decision trees, variable-order Markov modeling, optimum bit allocation, and computer graphics and image processing using quadtrees. A concentration on the first of these and a detailed analysis of variable-rate tree-structured vector quantization are provided. It is found that variable-rate tree-structured vector quantization outperforms not only the fixed-rate variety but also full-search vector quantization. The successive approximation character of variable-rate tree-structured vector quantization permits it to degrade gracefully if the rate is reduced at the encoder. This has applications to the problem of buffer overflow. >

[1]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[2]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[3]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[4]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[5]  James A. Anderson,et al.  A simple neural network generating an interactive memory , 1972 .

[6]  Kaoru Nakano,et al.  Associatron-A Model of Associative Memory , 1972, IEEE Trans. Syst. Man Cybern..

[7]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[8]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[9]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[10]  Robert M. Gray,et al.  Speech coding based upon vector quantization , 1980, ICASSP.

[11]  Robert M. Gray,et al.  Vector Quantizers and Predictive Quantizers for Gauss-Markov Sources , 1982, IEEE Trans. Commun..

[12]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[13]  Biing-Hwang Juang,et al.  An 800 bit/s vector quantization LPC vocoder , 1982 .

[14]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Hüseyin Abut,et al.  Full search and tree searched vector quantization of speech waveforms , 1982, ICASSP.

[16]  JORMA RISSANEN,et al.  A universal data compression system , 1983, IEEE Trans. Inf. Theory.

[17]  V. Cuperman,et al.  Vector quantization: A pattern-matching technique for speech coding , 1983, IEEE Communications Magazine.

[18]  Richard J. Higgins.  Electronics With Digital and Analog Integrated Circuits , 1983 .

[19]  Gérard D. Cohen,et al.  A nonconstructive upper bound on covering radius , 1983, IEEE Trans. Inf. Theory.

[20]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[21]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[22]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[23]  Yair Shoham,et al.  Efficient codebook allocation for an arbitrary set of vector quantizers , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  David G. Messerschmitt,et al.  Embedded coding of speech: A vector quantization approach , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  J. Makhoul,et al.  Vector quantization in speech coding , 1985, Proceedings of the IEEE.

[26]  Yaser S. Abu-Mostafa,et al.  Information capacity of the Hopfield model , 1985, IEEE Trans. Inf. Theory.

[27]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[28]  Allen Gersho,et al.  Application of a VLSI Vector Quantization Processor to Real-Time Speech Coding , 1986, IEEE J. Sel. Areas Commun..

[29]  Allen Gersho,et al.  A fast codebook search algorithm for nearest-neighbor pattern matching , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[30]  Alfred M. Bruckstein.  On "Soft" bit allocation , 1987, IEEE Trans. Acoust. Speech Signal Process..

[31]  Chia-Lung Yeh.  Color image-sequence compression using adaptive binary-tree vector quantization with codebook replenishment , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[32]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[33]  J. Smith,et al.  Vector quantizer architectures for speech and image coding , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[34]  Allen Gersho,et al.  Variable block-size image coding , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[35]  J. Keeler.  Information capacity of outer-product neural networks , 1987 .

[36]  J. Barrilleaux,et al.  Efficient vector quantization for color image encoding , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  Geoffrey E. Hinton,et al.  Connectionist Architectures for Artificial Intelligence , 1990, Computer.

[38]  Alan H. Barr,et al.  Accurate triangulations of deformed, intersecting surfaces , 1987, SIGGRAPH.

[39]  Bernard Hammer,et al.  Hierarchical encoding of image sequences using multistage vector quantization , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[40]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[41]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[42]  Robert M. Gray,et al.  A unified approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization , 1988, IEEE Trans. Inf. Theory.

[43]  Richard L. Baker,et al.  A VLSI chip set for real time vector quantization of image sequences , 1989 .