Receptive field optimisation and supervision of a fuzzy spiking neural network

This paper presents a supervised training algorithm that implements fuzzy reasoning on a spiking neural network. Neuron selectivity is facilitated using receptive fields that enable individual neurons to be responsive to certain spike train firing rates and behave in a similar manner as fuzzy membership functions. The connectivity of the hidden and output layers in the fuzzy spiking neural network (FSNN) is representative of a fuzzy rule base. Fuzzy C-Means clustering is utilised to produce clusters that represent the antecedent part of the fuzzy rule base that aid classification of the feature data. Suitable cluster widths are determined using two strategies; subjective thresholding and evolutionary thresholding respectively. The former technique typically results in compact solutions in terms of the number of neurons, and is shown to be particularly suited to small data sets. In the latter technique a pool of cluster candidates is generated using Fuzzy C-Means clustering and then a genetic algorithm is employed to select the most suitable clusters and to specify cluster widths. In both scenarios, the network is supervised but learning only occurs locally as in the biological case. The advantages and disadvantages of the network topology for the Fisher Iris and Wisconsin Breast Cancer benchmark classification tasks are demonstrated and directions of current and future work are discussed.

[1]  Ammar Belatreche,et al.  A Method for Supervised Training of Spiking Neural Networks , 2003 .

[2]  Eduardo D. Sontag,et al.  Neural Systems as Nonlinear Filters , 2000, Neural Computation.

[3]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[4]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[5]  Lovorka Pantic,et al.  Coincidence detection with dynamic synapses. , 2003, Network.

[6]  B. Katz,et al.  Quantal components of the end‐plate potential , 1954, The Journal of physiology.

[7]  Michael R. DeWeese,et al.  Neurobiology: Efficiency measures , 2006, Nature.

[8]  H. Barlow.  Summation and inhibition in the frog's retina , 1953, The Journal of physiology.

[9]  Jean-Pascal Pfister,et al.  Optimal Hebbian Learning: A Probabilistic Point of View , 2003, ICANN.

[10]  Andrzej J. Kasinski,et al.  Experimental Demonstration of Learning Properties of a New Supervised Learning Method for the Spiking Neural Networks , 2005, ICANN.

[11]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[12]  Michael Schmitt,et al.  Learning Temporally Encoded Patterns in Networks of Spiking Neurons , 2004, Neural Processing Letters.

[13]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[14]  Henry Markram,et al.  Coding of temporal information by activity-dependent synapses. , 2002, Journal of neurophysiology.

[15]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[16]  J. Deuchars,et al.  Synaptic interactions in neocortical local circuits: dual intracellular recordings in vitro. , 1997, Cerebral cortex.

[17]  Nancy Kopell,et al.  Synchronization in Networks of Excitatory and Inhibitory Neurons with Sparse, Random Connectivity , 2003, Neural Computation.

[18]  Anatol C. Kreitzer,et al.  Interplay between Facilitation, Depression, and Residual Calcium at Three Presynaptic Terminals , 2000, The Journal of Neuroscience.

[19]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[20]  Daniel Richardson,et al.  Linear Algebra for Time Series of Spikes , 2005, ESANN.

[21]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[22]  Jorge V. José,et al.  Inhibitory synchrony as a mechanism for attentional gain modulation , 2004, Journal of Physiology-Paris.

[23]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[24]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[25]  E. Fortune,et al.  Short-term synaptic plasticity as a temporal filter , 2001, Trends in Neurosciences.

[26]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[27]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[28]  Cornelius Glackin,et al.  Implementing Fuzzy Reasoning on a Spiking Neural Network , 2008, ICANN.

[29]  Andrzej Kasiński,et al.  Comparison of supervised learning methods for spike time coding in spiking neural networks , 2006 .

[30]  E. Cantu-Paz,et al.  An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Robert A. Legenstein,et al.  What Can a Neuron Learn with Spike-Timing-Dependent Plasticity? , 2005, Neural Computation.

[32]  Kenneth D. Miller,et al.  Adaptive filtering enhances information transmission in visual cortex , 2006, Nature.

[33]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[34]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[35]  Cornelius Glackin,et al.  Classification using a Fuzzy Spiking Neural Network , 2008 .

[36]  L. Abbott,et al.  Synaptic computation , 2004, Nature.

[37]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[38]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[39]  Jacques Sougné,et al.  A Learning Algorithm for Synfire Chains , 2000, NCPW.

[40]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[41]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[42]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[43]  Sridhar Narayan,et al.  Receptive field optimization for ensemble encoding , 2006, Neural Computing & Applications.

[44]  Robert M. French,et al.  Connectionist models of learning, development and evolution : Proceedings of the sixth neural computation and psychology workshop, Liège, Belgium, 16-18 september 2000 , 2001 .

[45]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[46]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[47]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[48]  Wolfgang Maass,et al.  Finding the Key to a Synapse , 2000, NIPS.

[49]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[50]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[51]  Charles F Stevens,et al.  Synaptic plasticity , 1998, Current Biology.