A Spiking Neuron Model: Applications and Learning

This paper presents a biologically inspired, hardware-realisable spiking neuron model, which we call the Temporal Noisy-Leaky Integrator (TNLI). The dynamic applications of the model as well as its applications in Computational Neuroscience are demonstrated and a learning algorithm based on postsynaptic delays is proposed. The TNLI incorporates temporal dynamics at the neuron level by modelling both the temporal summation of dendritic postsynaptic currents which have controlled delay and duration and the decay of the somatic potential due to its membrane leak. Moreover, the TNLI models the stochastic neurotransmitter release by real neuron synapses (with probabilistic RAMs at each input) and the firing times including the refractory period and action potential repolarisation. The temporal features of the TNLI make it suitable for use in dynamic time-dependent tasks like its application as a motion and velocity detector system presented in this paper. This is done by modelling the experimental velocity selectivity curve of the motion sensitive H1 neuron of the visual system of the fly. This application of the TNLI indicates its potential applications in artificial vision systems for robots. It is also demonstrated that Hebbian-based learning can be applied in the TNLI for postsynaptic delay training based on coincidence detection, in such a way that an arbitrary temporal pattern can be detected and recognised. The paper also demonstrates that the TNLI can be used to control the firing variability through inhibition; with 80% inhibition to concurrent excitation, firing at high rates is nearly consistent with a Poisson-type firing variability observed in cortical neurons. It is also shown with the TNLI, that the gain of the neuron (slope of its transfer function) can be controlled by the balance between inhibition and excitation, the gain being a decreasing function of the proportion of inhibitory inputs. Finally, in the case of perfect balance between inhibition and excitation, i.e. where the average input current is zero, the neuron can still fire as a result of membrane potential fluctuations. The firing rate is then determined by the average input firing rate. Overall this work illustrates how a hardware-realisable neuron model can capitalise on the unique computational capabilities of biological neurons.

[1]  W. Singer,et al.  Integrator or coincidence detector? The role of the cortical neuron revisited , 1996, Trends in Neurosciences.

[2]  Christof Koch,et al.  A Robust Analog VLSI Motion Sensor Based on the Visual System of the Fly , 1999, Auton. Robots.

[3]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Second Edition , 1988, Springer Series in Information Sciences.

[4]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[5]  Terrence J. Sejnowski,et al.  RAPID STATE SWITCHING IN BALANCED CORTICAL NETWORK MODELS , 1995 .

[6]  J. Jack,et al.  Electric current flow in excitable cells , 1975 .

[7]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[8]  Jianfeng Feng,et al.  Coefficient of variation of interspike intervals greater than 0.5. How and when? , 1999, Biological Cybernetics.

[9]  Guido Bugmann,et al.  Analysis of fluctuation-induced firing in the presence of inhibition , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[10]  S. W. Kuffler,et al.  From Neuron to Brain: A Cellular and Molecular Approach to the Function of the Nervous System , 1992 .

[11]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[12]  J. Buhmann,et al.  Influence of noise on the function of a “physiological” neural network , 1987, Biological Cybernetics.

[13]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[14]  Guido Bugmann,et al.  Multiplying with neurons: Compensation for irregular input spike trains by using time-dependent synaptic efficiencies , 2004, Biological Cybernetics.

[15]  W. Levick,et al.  Responses of cat retinal ganglion cells to brief flashes of light , 1970, The Journal of physiology.

[16]  Yves Frégnac,et al.  A tale of two spikes , 1999, Nature Neuroscience.

[17]  W. Holmes,et al.  A continuous cable method for determining the transient potential in passive dendritic trees of known geometry , 1986, Biological Cybernetics.

[18]  Rodney J. Douglas,et al.  Synchronization of Bursting Action Potential Discharge in a Model Network of Neocortical Neurons , 1991, Neural Computation.

[19]  Guido Bugmann,et al.  Role of Temporal Integration and Fluctuation Detection in the Highly Irregular Firing of a Leaky Integrator Neuron Model with Partial Reset , 1997, Neural Computation.

[20]  Erich Buchner,et al.  Behavioural Analysis of Spatial Vision in Insects , 1984 .

[21]  Mmh,et al.  Studies from the Rockefeller Institute for Medical Research , 1945 .

[22]  John G. Taylor,et al.  Discrete time leaky integrator network with synaptic noise , 1991, Neural Networks.

[23]  G Bugmann,et al.  Near Poisson-type firing produced by concurrent excitation and inhibition. , 2000, Bio Systems.

[24]  C. Koch,et al.  Synaptic background activity influences spatiotemporal integration in single pyramidal cells. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[25]  J. Leo van Hemmen,et al.  Modeling Synaptic Plasticity in Conjunction with the Timing of Pre- and Postsynaptic Action Potentials , 2000, Neural Computation.

[26]  John G. Taylor,et al.  Learning Probabilistic RAM Nets Using VLSI Structures , 1992, IEEE Trans. Computers.

[27]  Ken-ichi Hara,et al.  A generalized Hebbian rule for activity-dependent synaptic modifications , 2000, Neural Networks.

[28]  N. Franceschini,et al.  Real time visuomotor control: from flies to robots , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[29]  W. Rall Membrane potential transients and membrane time constant of motoneurons. , 1960, Experimental neurology.

[30]  W. Rall Cable theory for dendritic neurons , 1989 .

[31]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[32]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[33]  T G Clarkson,et al.  The pRAM: an adaptive VLSI chip , 1993, IEEE Trans. Neural Networks.

[34]  H. Tuckwell Linear cable theory and dendritic structure , 1988 .

[35]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[36]  Guido Bugmann,et al.  A temporal noisy-leaky integrator neuron constructed using pRAMs , 1992 .

[37]  John J. Hopfield,et al.  Neurons, Dynamics and Computation , 1994 .

[38]  B Horwitz,et al.  An analytical method for investigating transient potentials in neurons with branching dendritic trees. , 1981, Biophysical journal.

[39]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[40]  N. Franceschini,et al.  Early processing of colour and motion in a mosaic visual system. , 1985, Neuroscience research. Supplement : the official journal of the Japan Neuroscience Society.

[41]  Dendritic computation of direction selectivity by retinal ganglion cells. , 2000, Science.

[42]  W. Rall Theory of Physiological Properties of Dendrites , 1962, Annals of the New York Academy of Sciences.

[43]  Alexander Borst,et al.  Visuomotor coordination in flies , 1998 .

[44]  B Horwitz,et al.  Unequal diameters and their effects on time-varying voltages in branched neurons. , 1983, Biophysical journal.

[45]  Mark E. Nelson,et al.  A Mechanism for Neuronal Gain Control by Descending Pathways , 1994, Neural Computation.

[46]  Chris Christodoulou Temporal pattern detection and recognition using the temporal noisy-leaky integrator neuron model with the postsynaptic delays trained using Hebbian Learning , 1995 .

[47]  Nicolas Brunel,et al.  Global Spontaneous Activity and Local Structured (learned) Delay Activity in Cortex , 1995 .

[48]  J J Hopfield,et al.  Transforming neural computations and representing time. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[49]  J. Cowan,et al.  Transient potentials in dendritic systems of arbitrary geometry. , 1974, Biophysical journal.

[50]  B. Sakmann,et al.  A new cellular mechanism for coupling inputs arriving at different cortical layers , 1999, Nature.

[51]  Henry Markram,et al.  An Algorithm for Modifying Neurotransmitter Release Probability Based on Pre- and Postsynaptic Spike Timing , 2001, Neural Computation.

[52]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[53]  T. Poggio,et al.  Retinal ganglion cells: a functional interpretation of dendritic morphology. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[54]  Wolfgang Maass,et al.  Computation with spiking neurons , 2003 .

[55]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[56]  A. Hodgkin,et al.  The electrical constants of a crustacean nerve fibre , 1946, Proceedings of the Royal Society of London. Series B - Biological Sciences.

[57]  Werner Reichardt,et al.  Evaluation of optical motion information by movement detectors , 1987, Journal of Comparative Physiology A.

[58]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[59]  N. Franceschini,et al.  From insect vision to robot vision , 1992 .

[60]  Michael Jansen,et al.  Recognition and tracking of impulse patterns with delay adaptation in biology-inspired pulse processing neural net (BPN) hardware , 2004, Biological Cybernetics.

[61]  R. Shapley,et al.  Photoreception and Vision in Invertebrates , 1984, NATO ASI Series.

[62]  Mandyam V. Srinivasan,et al.  Motion detection in insect orientation and navigation , 1999, Vision Research.

[63]  W. Rall Branching dendritic trees and motoneuron membrane resistivity. , 1959, Experimental neurology.

[64]  Christof Koch,et al.  A simple algorithm for solving the cable equation in dendritic trees of arbitrary geometry , 1985, Journal of Neuroscience Methods.

[65]  Chris Christodoulou,et al.  Coefficient of Variation ( CV ) vs Mean Interspike Interval ( ISI ) curves : what do they tell us about the brain ? , 2000 .

[66]  J Rinzel,et al.  Transient response in a dendritic neuron model for current injected at one branch. , 1974, Biophysical journal.

[67]  Idan Segev,et al.  Compartmental models of complex neurons , 1989 .

[68]  B. Katz The release of neural transmitter substances , 1969 .

[69]  G. Ermentrout,et al.  Analysis of neural excitability and oscillations , 1989 .

[70]  John G. Taylor,et al.  Storing temporal sequences , 1991, Neural Networks.

[71]  Christof Koch,et al.  Neuromorphic vision chips , 1996 .

[72]  B. Hassenstein,et al.  Systemtheoretische Analyse der Zeit-, Reihenfolgen- und Vorzeichenauswertung bei der Bewegungsperzeption des Rüsselkäfers Chlorophanus , 1956 .

[73]  B. Sakmann,et al.  Active propagation of somatic action potentials into neocortical pyramidal cell dendrites , 1994, Nature.

[74]  L. Abbott,et al.  Model neurons: From Hodgkin-Huxley to hopfield , 1990 .

[75]  B. Walmsley,et al.  Effects of spatial and temporal dispersion of synaptic input on the time course of synaptic potentials. , 1989, Journal of neurophysiology.

[76]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[77]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[78]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[79]  Johannes M. Zanker,et al.  Speed tuning in elementary motion detectors of the correlation type , 1999, Biological Cybernetics.

[80]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[81]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[82]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[83]  T. Poggio,et al.  A New Approach to Synaptic Interactions , 1978 .

[84]  Alexa Riehle,et al.  Directionally Selective Motion Detection by Insect Neurons , 1989 .

[85]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[86]  C. Wilson,et al.  Passive cable properties of dendritic spines and spiny neurons , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[87]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[88]  S. Finkbeiner Mathematical Aspects of Hodgkin-Huxley Neural Theory , 1988, The Yale Journal of Biology and Medicine.

[89]  J J Hopfield,et al.  Rapid local synchronization of action potentials: toward computation with coupled integrate-and-fire neurons. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[90]  Wulfram Gerstner,et al.  Spiking neurons , 1999 .

[91]  C. Koch,et al.  Methods in Neuronal Modeling: From Ions to Networks , 1998 .

[92]  Tomaso Poggio,et al.  Computations in the vertebrate retina: gain enhancement, differentiation and motion discrimination , 1986, Trends in Neurosciences.

[93]  R. Lorente de Nó,et al.  Contribution to the mathematical theory of the electrotonus. , 1947, Studies from the Rockefeller institute for medical research. Reprints. Rockefeller Institute for Medical Research.

[94]  Sj Redman Monosynaptic Transmission in the Spinal Cord , 1986 .

[95]  Taylor,et al.  Spatiotemporal pattern processing in a compartmental-model neuron. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[96]  R. FitzHugh Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[97]  J. Taylor Spontaneous behaviour in neural networks. , 1972, Journal of theoretical biology.

[98]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[99]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[100]  Martin Egelhaaf,et al.  Neural Mechanisms of Visual Course Control in Insects , 1989 .

[101]  J. Simpson THE RELEASE OF NEURAL TRANSMITTER SUBSTANCES , 1969 .

[102]  S. Redman Quantal analysis of synaptic potentials in neurons of the central nervous system. , 1990, Physiological reviews.

[103]  E. Buchner Elementary movement detectors in an insect visual system , 1976, Biological Cybernetics.

[104]  S. Levin Lectu re Notes in Biomathematics , 1983 .

[105]  Jianfeng Feng,et al.  Impact of temporal variation and the balance between excitation and inhibition on the output of the perfect integrate-and-fire model , 1998, Biological Cybernetics.

[106]  Davide Badoni,et al.  Spike-Driven Synaptic Plasticity: Theory, Simulation, VLSI Implementation , 2000, Neural Computation.

[107]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[108]  P G Nelson,et al.  Active and inactive central synapses in cell culture. , 1986, Journal of neurophysiology.

[109]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[110]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[111]  G Indiveri,et al.  Neuromorphic Vision Sensors , 2000, Science.

[112]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[113]  Wulfram Gerstner,et al.  A developmental learing rule for coincidence tuning in the barn owl auditory system , 1997 .

[114]  R. Hardie,et al.  Facets of Vision , 1989, Springer Berlin Heidelberg.

[115]  H. Tuckwell Introduction to Theoretical Neurobiology: Linear Cable Theory and Dendritic Structure , 1988 .

[116]  C. Stevens,et al.  Input synchrony and the irregular firing of cortical neurons , 1998, Nature Neuroscience.

[117]  Örjan Ekeberg,et al.  A computer based model for realistic simulations of neural networks , 1991, Biological Cybernetics.

[118]  James M. Bower,et al.  Computational Neuroscience: Trends in Research , 1996 .

[119]  Jonathan Bell,et al.  Introduction to Theoretical Neurobiology. Volume 1: Linear Cable Theory and Dendritic Structure. Volume 2: Nonlinear and Stochastic Theories (Henry C. Tuckwell) , 1990, SIAM Rev..

[120]  Guido Bugmann,et al.  Coefficient of variation vs. mean interspike interval curves: What do they tell us about the brain? , 2001, Neurocomputing.

[121]  A. Hodgkin,et al.  The frequency of nerve action potentials generated by applied currents , 1967, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[122]  T. Pun,et al.  Figure-ground separation: evidence for asynchronous processing in visual perception? , 1991 .

[123]  Idan Segev,et al.  The Impact of Parallel Fiber Background Activity on the Cable Properties of Cerebellar Purkinje Cells , 1992, Neural Computation.

[124]  Wilfrid Rall,et al.  Theoretical significance of dendritic trees for neuronal input-output relations , 1964 .