Kernel-Based Equiprobabilistic Topographic Map Formation

We introduce a new unsupervised competitive learning rule, the kernel-based maximum entropy learning rule (kMER), which performs equiprobabilistic topographic map formation in regular, fixed-topology lattices, for use with nonparametric density estimation as well as nonparametric regression analysis. The receptive fields of the formal neurons are overlapping radially symmetric kernels, compatible with radial basis functions (RBFs); but unlike other learning schemes, the radii of these kernels do not have to be chosen in an ad hoc manner: the radii are adapted to the local input density, together with the weight vectors that define the kernel centers, so as to produce maps of which the neurons have an equal probability to be active (equiprobabilistic maps). Both an online and a batch version of the learning rule are introduced, which are applied to nonparametric density estimation and regression, respectively. The application envisaged is blind source separation (BSS) from nonlinear, noisy mixtures.

[1]  L. Breiman,et al.  Variable Kernel Estimates of Multivariate Densities , 1977 .

[2]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[3]  B. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[4]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[5]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[6]  Steven J. Nowlan,et al.  Maximum Likelihood Competitive Learning , 1989, NIPS.

[7]  Olli Simula,et al.  Combining linear equalization and self-organizing adaptation in dynamic discrete-signal detection , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[9]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[10]  David Lowe,et al.  What have neural networks to offer statistical pattern processing? , 1991, Optics & Photonics.

[11]  Helge Ritter.  Asymptotic level density for a class of vector quantization processes , 1991, IEEE Trans. Neural Networks.

[12]  Stephen P. Luttrell.  Code vector density in topographic mappings: Scalar case , 1991, IEEE Trans. Neural Networks.

[13]  Bernd Fritzke.  Growing Cell Structures – a Self-organizing Network in k Dimensions , 1992 .

[14]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[15]  Dominique Martinez,et al.  On an Unsupervised Learning Rule for Scalar Quantization following the Maximum Entropy Principle , 1993, Neural Computation.

[16]  Jenq-Neng Hwang,et al.  Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[17]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[18]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[19]  M. Hulle.  Globally-ordered topology-preserving maps achieved with a learning rule performing local weight updates only , 1995 .

[20]  Vladimir Cherkassky,et al.  Self-Organization as an Iterative Kernel Smoothing Process , 1995, Neural Computation.

[21]  Eric Moreau,et al.  High order contrasts for self-adaptive source separation criteria for complex source separation , 1996 .

[22]  Ralf Der,et al.  Controlling the Magnification Factor of Self-Organizing Feature Maps , 1996, Neural Computation.

[23]  A. Hyvärinen,et al.  Nonlinear Blind Source Separation by Self-Organizing Maps , 1996 .

[24]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[25]  Marc M. Van Hulle,et al.  The Formation of Topographic Maps That Maximize the Average Mutual Information of the Output Responses to Noiseless Input Signals , 1997, Neural Computation.

[26]  Marc M. Van Hulle,et al.  Topology-preserving Map Formation Achieved with a Purely Local Unsupervised Competitive Learning Rule , 1997, Neural Networks.

[27]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[28]  Juan K. Lin,et al.  Faithful Representation of Separable Distributions , 1997, Neural Computation.

[29]  K. Obermayer,et al.  PHASE TRANSITIONS IN STOCHASTIC SELF-ORGANIZING MAPS , 1997 .

[30]  Marc M. Van Hulle.  Nonparametric density estimation and regression achieved with topographic maps maximizing the information-theoretic entropy of their outputs , 1997, Biological Cybernetics.

[31]  Rodolfo Zunino,et al.  Efficient training of neural gas vector quantizers with analog circuit implementation , 1999 .

[32]  Sandro Ridella,et al.  Circular backpropagation networks embed vector quantization , 1999, IEEE Trans. Neural Networks.