Spike-Rate Perceptrons

According to the diffusion approximation, we present a more biologically plausible so-called spike-rate perceptron based on IF model with renewal process inputs, which employs both first and second statistical representation, i.e. the means, variances and correlations of the synaptic input. We first identify the input-output relationship of the spike-rate model and apply an error minimization technique to train the model. We then show that it is possible to train these networks with a mathematically derived learning rule. We show through various examples that such perceptron, even a single neuron, is able to perform various complex non-linear tasks like the XOR problem. Here our perceptrons offer a significant advantage over classical models, in that they include both the mean and the variance of the input signal. Our ultimate purpose is to open up the possibility of carrying out a random computation in neuronal networks, by introducing second order statistics in computations.

[1]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[2]  X. Wang,et al.  Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory , 1999, The Journal of Neuroscience.

[3]  R. Stein.  Some models of neuronal variability. , 1967, Biophysical journal.

[4]  Henry C. Tuckwell,et al.  Introduction to theoretical neurobiology , 1988 .

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  J. Hunter.  Renewal theory in two dimensions: asymptotic results , 1974, Advances in Applied Probability.

[7]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[8]  Jianfeng Feng,et al.  Computational neuroscience , 1986, Behavioral and Brain Sciences.

[9]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[10]  Jianfeng Feng,et al.  Is the integrate-and-fire model good enough?--a review , 2001, Neural Networks.

[11]  A. Treves.  Mean-field analysis of neuronal spike dynamics , 1993 .

[12]  P. Matthews.  Relationship of firing intervals of human motor units to the trajectory of post‐spike after‐hyperpolarization and synaptic noise. , 1996, The Journal of physiology.

[13]  Jianfeng Feng,et al.  Optimal control of neuronal activity. , 2003, Physical review letters.

[14]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[15]  H. Tuckwell.  Introduction to Theoretical Neurobiology: Linear Cable Theory and Dendritic Structure , 1988 .

[16]  J. Orbach.  Principles of Neurodynamics. Perceptrons and the Theory of Brain Mechanisms. , 1962 .

[17]  L. Abbott,et al.  Synaptic Depression and Cortical Gain Control , 1997, Science.

[18]  M. Mattia,et al.  Population dynamics of interacting spiking neurons. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[19]  Jianfeng Feng,et al.  Training the integrate-and-fire model with the informax principle: I , 2002 .

[20]  Jianfeng Feng,et al.  Responses of Magnocellular Neurons to Osmotic Stimulation Involves Coactivation of Excitatory and Inhibitory Input: An Experimental and Theoretical Analysis , 2001, The Journal of Neuroscience.