论文信息 - Equilibrium Properties of Offline LVQ

Equilibrium Properties of Offline LVQ

The statistical physics analysis of offline learning is applied to cost function based learning vector quantization (LVQ) schemes. Typical learning behavior is obtained from a model with data drawn from high dimensional Gaussian mixtures and a system of two or three competing prototypes. The analytic approach becomes exact in the limit of high training temperature. We study two cost function related LVQ algorithms and the influence of an appropriate weight decay. In our findings, learning from mistakes (LFM) achieves poor generalization ability, while a limiting case of generalized LVQ (GLVQ), termed LVQ+/-, displays much better performance with a properly chosen weight decay.

Michael Biehl | Barbara Hammer | Aree Witoelar

[1] C. Van Den Broeck,et al. Analysing Cluster Formation by Replica Method , 1995 .

[2] Michael Biehl,et al. Phase transitions in soft-committee machines , 1998, cond-mat/9805182.

[3] T. Watkin,et al. THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[4] Christian Van den Broeck,et al. Statistical Mechanics of Learning , 2001 .

[5] Michael Biehl,et al. Learning dynamics and robustness of vector quantization and neural gas , 2008, Neurocomputing.

[6] Atsushi Sato,et al. Generalized Learning Vector Quantization , 1995, NIPS.

[7] Michael Biehl,et al. Weight-decay induced phase transitions in multilayer neural networks , 1999, cond-mat/9901179.

[8] Sompolinsky,et al. Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[9] Klaus Obermayer,et al. Soft Learning Vector Quantization , 2003, Neural Computation.

[10] Michael Biehl,et al. Phase transitions in vector quantization and neural gas , 2009, Neurocomputing.

[11] Michael Biehl,et al. Dynamics and Generalization Ability of LVQ Algorithms , 2007, J. Mach. Learn. Res..