The Formation of Topographic Maps That Maximize the Average Mutual Information of the Output Responses to Noiseless Input Signals

This article introduces an extremely simple and local learning rule for to pographic map formation. The rule, called the maximum entropy learning rule (MER), maximizes the unconditional entropy of the map's output for any type of input distribution. The aim of this article is to show that MER is a viable strategy for building topographic maps that maximize the average mutual information of the output responses to noiseless input signals when only input noise and noise-added input signals are available.

[1]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[2]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[3]  Alan V. Oppenheim,et al.  Digital Signal Processing , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Dirk Van Compernolle,et al.  A family of MLP based nonlinear spectral estimators for noise reduction , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Ralph Linsker,et al.  How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals , 1989, Neural Computation.

[6]  Gerhard Rigoll,et al.  Maximum mutual information neural networks for hybrid connectionist-HMM speech recognition systems , 1994, IEEE Trans. Speech Audio Process..

[7]  Terence D. Sanger,et al.  An Optimality Principle for Unsupervised Learning , 1988, NIPS.

[8]  T. Adalı,et al.  A maximum partial likelihood framework for channel equalization by distribution learning , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[9]  Joseph J. Atick,et al.  Predicting Ganglion and Simple Cell Receptive Field Organizations , 1991, Int. J. Neural Syst..

[10]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[11]  Marc M. Van Hulle,et al.  Nonparametric regression analysis achieved with topographic maps developed in combination with projection pursuit learning: an application to density estimation and adaptive filtering of grey-scale images , 1997, IEEE Trans. Signal Process..

[12]  W. Yang,et al.  A robust backward adaptive quantizer , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[13]  Ralph Linsker,et al.  Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network , 1992, Neural Computation.

[14]  A. Hyvärinen,et al.  Nonlinear Blind Source Separation by Self-Organizing Maps , 1996 .

[15]  M. Hulle.  Kernel-Based Equiprobabilistic Topographic Map Formation , 1998, Neural Computation.

[16]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.