Convergence properties of a class of learning vector quantization algorithms

A mathematical analysis of a class of learning vector quantization (LVQ) algorithms is presented. Using an appropriate time-coordinate transformation, we show that the LVQ algorithms under consideration can be transformed into linear time-varying stochastic difference equations. Using this fact, we apply stochastic Lyapunov stability arguments, and we prove that the LVQ algorithms under consideration do indeed converge, provided that some appropriate conditions hold.

[1]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[2]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[3]  Tamio Shimizu,et al.  A Stochastic Approximation Method for Optimization Problems , 1969, Journal of the ACM.

[4]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[5]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[6]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[7]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[8]  Robert M. Gray,et al.  Global convergence and empirical consistency of the generalized Lloyd algorithm , 1986, IEEE Trans. Inf. Theory.

[9]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[10]  M. Hersh.  Recursive estimation and control for stochastic systems. H. F. Chen, Wiley. New York, 1985. No. of pages: 288. Price: £41.40. ISBN 0471‐81566‐7. , 1987 .

[11]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[12]  Stephen P. Luttrell,et al.  Derivation of a class of training algorithms , 1990, IEEE Trans. Neural Networks.

[13]  David M. Clark,et al.  A convergence theorem for Grossberg learning , 1990, Neural Networks.

[14]  John S. Baras,et al.  Convergence of Kohonen's learning vector quantization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[15]  B. V. K. Vijaya Kumar,et al.  Competitive learning with generalized winner-take-all activation , 1992, IEEE Trans. Neural Networks.

[16]  Zhen-Ping Lo,et al.  Analysis of the convergence properties of topology preserving neural networks , 1993, IEEE Trans. Neural Networks.

[17]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[18]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[19]  Michael D. Lemmon,et al.  Topologically ordered competitive sampling , 1994, Neural Networks.

[20]  G. Lugosi,et al.  Consistency of Data-driven Histogram Methods for Density Estimation and Classification , 1996 .