Competitive learning and soft competition for vector quantizer design

The authors provide a convergence analysis for the Kohonen learning algorithm (KLA) with respect to vector quantizer (VQ) optimality criteria and introduce a stochastic relaxation technique which produces the global minimum but is computationally expensive. By incorporating the principles of the stochastic approach into the KLA, a deterministic VQ design algorithm, the soft competition scheme (SCS), is introduced. Experimental results are presented where the SCS consistently provided better codebooks than the generalized Lloyd algorithm (GLA), even when the same computation time was used for both algorithms. The SCS may therefore prove to be a valuable alternative to the GLA for VQ design. >

[1]  Nasser M. Nasrabadi,et al.  Vector quantization of images based upon the Kohonen self-organizing feature maps , 1988, ICNN.

[2]  Allen Gersho,et al.  On the structure of vector quantizers , 1982, IEEE Trans. Inf. Theory.

[3]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[4]  Alberto L. Sangiovanni-Vincentelli,et al.  Efficient Parallel Learning Algorithms for Neural Networks , 1988, NIPS.

[5]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[6]  Pao-Chi Chang,et al.  Gradient algorithms for designing predictive vector quantizers , 1986, IEEE Trans. Acoust. Speech Signal Process..

[7]  A. Gersho.  Adaptaive vector quantization , 1986 .

[8]  Kenneth Zeger.  Corrections to 'Gradient algorithms for designing predictive vector quantizers' , 1991, IEEE Trans. Signal Process..

[9]  Nasser M. Nasrabadi,et al.  Vector quantization of images based upon the Kohonen self-organizing feature maps , 1988, IEEE 1988 International Conference on Neural Networks.

[10]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[11]  Allen Gersho,et al.  Adaptive vector quantization , 1992 .

[12]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[13]  E. Eweda,et al.  Convergence of an adaptive linear estimation algorithm , 1984 .

[14]  Allen Gersho,et al.  The Boltzmann Perceptron Network: A soft classifier , 1990, Neural Networks.

[15]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[16]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[17]  David J. Burr,et al.  Experiments on neural net recognition of spoken and written text , 1988, IEEE Trans. Acoust. Speech Signal Process..

[18]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[19]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[20]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[21]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[22]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[23]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[24]  R. Linggard,et al.  Neural arrays for speech recognition , 1990 .

[25]  Hervé Bourlard,et al.  Speech dynamics and recurrent neural networks , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[26]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[27]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[28]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Robert M. Gray,et al.  Multiple local optima in vector quantizers , 1982, IEEE Trans. Inf. Theory.

[30]  K. Schulten,et al.  Kohonen's self-organizing maps: exploring their computational capabilities , 1988, IEEE 1988 International Conference on Neural Networks.

[31]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[32]  Allen Gersho,et al.  Globally optimal vector quantizer design by stochastic relaxation , 1992, IEEE Trans. Signal Process..