Topographic map formation by maximizing unconditional entropy: a plausible strategy for "online" unsupervised competitive learning and nonparametric density estimation

An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of "online" learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm.

[1]  Peter Seitz,et al.  Minimum class entropy: A maximum information approach to layered networks , 1989, Neural Networks.

[2]  Yeshaiahu Fainman,et al.  A learning law for density estimation , 1994, IEEE Trans. Neural Networks.

[3]  V. Nollau.  Kushner, H. J./Clark, D. S., Stochastic Approximation Methods for Constrained and Unconstrained Systems. (Applied Mathematical Sciences 26). Berlin‐Heidelberg‐New York, Springer‐Verlag 1978. X, 261 S., 4 Abb., DM 26,40. US $ 13.20 , 1980 .

[4]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[5]  Sheldon B. Gardner,et al.  Application of neural network algorithms and architectures to correlation/tracking and identification , 1987 .

[6]  Dominique Martinez,et al.  Generalized boundary adaptation rule for minimizing rth power law distortion in high resolution quantization , 1995, Neural Networks.

[7]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[8]  Klaus Pawelzik,et al.  Quantifying the neighborhood preservation of self-organizing feature maps , 1992, IEEE Trans. Neural Networks.

[9]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[10]  Ray H. White,et al.  Competitive hebbian learning: Algorithm and demonstrations , 1992, Neural Networks.

[11]  Ralph Linsker,et al.  How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals , 1989, Neural Computation.

[12]  Gerhard Rigoll,et al.  Maximum mutual information neural networks for hybrid connectionist-HMM speech recognition systems , 1994, IEEE Trans. Speech Audio Process..

[13]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[14]  Dominique Martinez,et al.  On a novel unsupervised competitive learning algorithm for scalar quantization , 1994, IEEE Trans. Neural Networks.

[15]  Halbert White,et al.  Parametric Statistical Estimation with Artificial Neural Networks: A Condensed Discussion , 1994 .

[16]  Kurt Hornik,et al.  Convergence analysis of local feature extraction algorithms , 1992, Neural Networks.