Synaptic plasticity in spiking neural networks (SP2INN): a system approach

In this paper, we present a digital system called (SP/sup 2/INN) for simulating very large-scale spiking neural networks (VLSNNs) comprising, e.g., 1000000 neurons with several million connections in total. SP/sup 2/INN makes it possible to simulate VLSNN with features such as synaptic short term plasticity, long term plasticity as well as configurable connections. For such VLSNN the computation of the connectivity including the synapses is the main challenging task besides computing the neuron model. We describe the configurable neuron model of SP/sup 2/INN, before we focus on the computation of the connectivity. Within SP/sup 2/INN, connectivity parameters are stored in an external memory, while the actual connections are computed online based on defined connectivity rules. The communication between the SP/sup 2/INN processor and the external memory represents a bottle-neck for the system performance. We show this problem is handled efficiently by introducing a tag scheme and a target-oriented addressing method. The SP/sup 2/INN processor is described in a high-level hardware description language. We present its implementation in a 0.35 /spl mu/m CMOS technology, but also discuss advantages and drawbacks of implementing it on a field programmable gate array.

[1]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[2]  A S French,et al.  A flexible neural analog using integrated circuits. , 1970, IEEE transactions on bio-medical engineering.

[3]  H. T. Kung,et al.  A Regular Layout for Parallel Adders , 1982, IEEE Transactions on Computers.

[4]  Christof Koch,et al.  Modeling the mammalian visual system , 1989 .

[5]  M. Arndt,et al.  A neural network for feature linking via synchronous activity: Results from cat visual cortex and from simulations , 1989 .

[6]  P. Dicke,et al.  Feature linking via stimulus-evoked oscillations: experimental results from cat visual cortex and functional implications from a network model , 1989, International 1989 Joint Conference on Neural Networks.

[7]  H. Klar,et al.  Cascadable digital emulator IC for 16 biological neurons , 1993, 1993 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[8]  David A. Patterson,et al.  Computer Architecture - A Quantitative Approach, 5th Edition , 1996 .

[9]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[10]  Heinrich Klar,et al.  A SIMD/dataflow architecture for a neurocomputer for spike-processing neural networks (NESPINN) , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[11]  Heinrich Klar,et al.  On-Line Hebbian Learning for Spiking Neurons: Architecture of the Weight-Unit of NESPINN , 1997, ICANN.

[12]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[13]  L. Abbott,et al.  A Quantitative Description of Short-Term Plasticity at Excitatory Synapses in Layer 2/3 of Rat Primary Visual Cortex , 1997, The Journal of Neuroscience.

[14]  H. Markram,et al.  Information Processing with Frequency-Dependent Synaptic Connections , 1998, Neurobiology of Learning and Memory.

[15]  G. Hartmann,et al.  A flexible hardware architecture for online Hebbian learning in the sender-oriented PCNN-neurocomputer Spike 128 K , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[16]  U. Ruckert,et al.  ParSPIKE-a parallel DSP-accelerator for dynamic simulation of large spiking neural networks , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[17]  V. Bringuier,et al.  Horizontal propagation of visual activity in the synaptic integration field of area 17 neurons. , 1999, Science.

[18]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[19]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[20]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[21]  Henry Markram,et al.  An Algorithm for Modifying Neurotransmitter Release Probability Based on Pre- and Postsynaptic Spike Timing , 2001, Neural Computation.

[22]  S. Niculescu.  Delay Effects on Stability: A Robust Control Approach , 2001 .

[23]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[24]  Alan F. Murray,et al.  Citcuits for VLSI Implementation of Temporally Asymmetric Hebbian Learning , 2001, NIPS.

[25]  Tobi Delbrück,et al.  Orientation-Selective aVLSI Spiking Neurons , 2001, NIPS.

[26]  Ulrich Rückert,et al.  Implementation of artificial neural networks on a reconfigurable hardware accelerator , 2002, Proceedings 10th Euromicro Workshop on Parallel, Distributed and Network-based Processing.

[27]  Dean V. Buonomano,et al.  Mechanisms and significance of spike-timing dependent plasticity , 2002, Biological Cybernetics.

[28]  Patrick D. Roberts,et al.  Spike timing dependent synaptic plasticity in biological systems , 2002, Biological Cybernetics.

[29]  Tim Schönauer,et al.  NeuroPipe-Chip: A digital neuro-processor for spiking neural networks , 2002, IEEE Trans. Neural Networks.

[30]  Heinrich Klar,et al.  Image preprocessing with dynamic synapses , 2003, Neural Computing & Applications.

[31]  Wulfram Gerstner,et al.  Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns , 1993, Biological Cybernetics.