Simulation of networks of spiking neurons: A review of tools and strategies

We review different aspects of the simulation of spiking neural networks. We start by reviewing the different types of simulation strategies and algorithms that are currently implemented. We next review the precision of those simulation strategies, in particular in cases where plasticity depends on the exact timing of the spikes. We overview different simulators and simulation environments presently available (restricted to those freely available, open source and documented). For each simulation tool, its advantages and pitfalls are reviewed, with an aim to allow the reader to identify which simulator is appropriate for a given task. Finally, we provide a series of benchmark simulations of different types of networks of spiking neurons, including Hodgkin–Huxley type, integrate-and-fire models, interacting with current-based or conductance-based synapses, using clock-driven or event-driven integration strategies. The same set of models are implemented on the different simulators, and the codes are made available. The ultimate goal of this review is to provide a resource to facilitate identifying the appropriate integration strategy and simulation tool to use for a given modeling problem related to spiking neural networks.

[1]  George L.-T. Chiu,et al.  Overview of the Blue Gene/L system architecture , 2005, IBM J. Res. Dev..

[2]  Terrence J. Sejnowski,et al.  An Efficient Method for Computing Synaptic Conductances Based on a Kinetic Model of Receptor Binding , 1994, Neural Computation.

[3]  Dominique Martinez,et al.  Event-Driven Simulations of Nonlinear Integrate-and-Fire Neurons , 2007, Neural Computation.

[4]  Joel L. Davis,et al.  An Introduction to Neural and Electronic Networks , 1995 .

[5]  R.M. Fujimoto,et al.  Parallel and distributed simulation systems , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[6]  Thomas K. Berger,et al.  Heterogeneity in the pyramidal network of the medial prefrontal cortex , 2006, Nature Neuroscience.

[7]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Honeycutt,et al.  Stochastic Runge-Kutta algorithms. I. White noise. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[9]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[10]  Rick Stevens,et al.  Large Neural Simulations on Large Parallel Computers , 2005 .

[11]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[12]  Frederick C. Harris,et al.  COMPUTATIONAL MODEL OF INTERACTING BRAIN NETWORKS: 435 , 2004 .

[13]  Nicholas Carriero,et al.  Linda in context , 1989, CACM.

[14]  Stelios C. Orphanoudakis,et al.  Modelling reduced excitability in aged CA1 neurons as a calcium-dependent process , 2005, Neurocomputing.

[15]  Bijoy K. Ghosh,et al.  Propagating Waves in Visual Cortex: A Large-Scale Model of Turtle Visual Cortex , 2003, Journal of Computational Neuroscience.

[16]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[17]  Nicholas T. Carnevale,et al.  Discrete event simulation in the NEURON environment , 2004, Neurocomputing.

[18]  Michael L. Hines,et al.  The Role of Distal Dendritic Gap Junctions in Synchronization of Mitral Cell Axonal Output , 2005, Journal of Computational Neuroscience.

[19]  Idan Segev,et al.  Depressed responses of facilitatory synapses. , 2005, Journal of neurophysiology.

[20]  Andrew D. Brown,et al.  Discrete simulation of large aggregates of neurons , 2002, Neurocomputing.

[21]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[22]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[23]  Matthew A. Wilson,et al.  The simulation of large-scale neural networks , 1989 .

[24]  Florentin Wörgötter,et al.  Employing the Z-Transform to Optimize the Calculation of the Synaptic Conductance of NMDA and Other Synaptic Channels in Network Simulations , 1998, Neural Computation.

[25]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[26]  Etienne Hugues,et al.  Contour Detection by Synchronization of Integrate-and-Fire Neurons , 2002, Biologically Motivated Computer Vision.

[27]  Armen Saghatelyan,et al.  Activity-Dependent Adjustments of the Inhibitory Network in the Olfactory Bulb following Early Postnatal Deprivation , 2005, Neuron.

[28]  M Hines,et al.  A program for simulation of nerve equations with branching geometries. , 1989, International journal of bio-medical computing.

[29]  Frederick C. Harris,et al.  Remote-neocortex control of robotic search and threat identification , 2004, Robotics Auton. Syst..

[30]  Pierre Kornprobst,et al.  From Light to Spikes: a Large-Scale Retina Simulator , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[31]  Michele Giugliano,et al.  Event-Driven Simulation of Spiking Neurons with Stochastic Dynamics , 2003, Neural Computation.

[32]  Örjan Ekeberg,et al.  Large neural network simulations on multiple hardware platforms , 1997 .

[33]  U. Bhalla,et al.  Emergent properties of networks of biological signaling pathways. , 1999, Science.

[34]  John Kenyon,et al.  Accurate dynamical models of interneuronal GABAergic channel physiologies , 2005, Neurocomputing.

[35]  Ronan G. Reilly,et al.  APPROACHES TO EFFICIENT SIMULATION WITH SPIKING NEURAL NETWORKS , 2004 .

[36]  Frederick C. Harris,et al.  Design and Implementation of a Web Portal for a NeoCortical Simulator , 2002, Computers and Their Applications.

[37]  Padraig Gleeson,et al.  XML for Model Specification in Neuroscience An Introduction and Workshop Summary , 2005 .

[38]  Örjan Ekeberg,et al.  Large Neural Network Simulations on Multiple Hardware Platforms , 1997, Journal of Computational Neuroscience.

[39]  T. Makino.  A Discrete-Event Neural Network Simulator for General Neuron Models , 2003, Neural Computing & Applications.

[40]  Michael L. Hines,et al.  The NEURON Book , 2006 .

[41]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[42]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[43]  Susan R. Barry.  Neurons In Action: Computer Simulations with NeuroLab , 2004, Journal of Undergraduate Neuroscience Education.

[44]  Bernard P. Zeigler,et al.  Theory of Modeling and Simulation: Integrating Discrete Event and Continuous Complex Dynamic Systems , 2000 .

[45]  Anders Lansner,et al.  A hemicord locomotor network of excitatory interneurons: a simulation study , 2007, Biological Cybernetics.

[46]  Daniel D. Lee,et al.  Equilibrium properties of temporally asymmetric Hebbian plasticity. , 2000, Physical review letters.

[47]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[48]  Michele Giugliano,et al.  Synthesis of Generalized Algorithms for the Fast Computation of Synaptic Conductances with Markov Kinetic Models in Large Network Simulations , 2000, Neural Computation.

[49]  R. Azouz.  Dynamic spatiotemporal synaptic integration in cortical neurons: neuronal gain, revisited. , 2005, Journal of neurophysiology.

[50]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[51]  J. Bower,et al.  An active membrane model of the cerebellar Purkinje cell. I. Simulation of current clamps in slice. , 1994, Journal of neurophysiology.

[52]  Wulfram Gerstner,et al.  Mathematical formulations of Hebbian learning , 2002, Biological Cybernetics.

[53]  H. Tuckwell.  Introduction to Theoretical Neurobiology: Linear Cable Theory and Dendritic Structure , 1988 .

[54]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[55]  T. Sejnowski,et al.  Homeostatic synaptic plasticity can explain post-traumatic epileptogenesis in chronically isolated neocortex. , 2005, Cerebral cortex.

[56]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[57]  Michael Affenzeller,et al.  DEVS Simulation of Spiking Neural Networks , 2003 .

[58]  Robert E. Tarjan,et al.  Self-adjusting binary trees , 1983, STOC.

[59]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[60]  B. P. Ziegler,et al.  Theory of Modeling and Simulation , 1976 .

[61]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[62]  Raquel Seiça,et al.  Disruption of Mitochondrial Calcium Homeostasis after Chronic α-Naphthylisothiocyanate Administration: Relevance for Cholestasis , 2002, Journal of Investigative Medicine.

[63]  Bernard P. Zeigler,et al.  Devs Formalism and Methodology: Unity of Conception/diversity of Application , 1993, Proceedings of 1993 Winter Simulation Conference - (WSC '93).

[64]  Manuel A. Sánchez-Montañés.  Strategies for the Optimization of Large Scale Networks of Integrate and Fire Neurons , 2001, IWANN.

[65]  Satish K. Tripathi,et al.  Parallel and distributed simulation of discrete event systems , 1994 .

[66]  Tim Schönauer,et al.  Digital simulation of spiking neural networks , 1999 .

[67]  Fiona E. N. LeBeau,et al.  Single-column thalamocortical network model exhibiting gamma oscillations, sleep spindles, and epileptogenic bursts. , 2005, Journal of neurophysiology.

[68]  Romain Brette,et al.  Exact Simulation of Integrate-and-Fire Models with Synaptic Conductances , 2006, Neural Computation.

[69]  Nigel H. Goddard,et al.  Towards NeuroML: model description methods for collaborative modelling in neuroscience. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[70]  Henry Markram,et al.  Computer models and analysis tools for neural microcircuits , 2003 .

[71]  R. Morris.  Foundations of cellular neurophysiology , 1996 .

[72]  Anders Lansner,et al.  Scaling effects in a model of the olfactory bulb , 2007, Neurocomputing.

[73]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[74]  Y. Dan,et al.  Spike-timing-dependent synaptic modification induced by natural spike trains , 2002, Nature.

[75]  Alain Destexhe,et al.  How much can we trust neural simulation strategies? , 2007, Neurocomputing.

[76]  Anders Lansner,et al.  Burst dynamics under mixed NMDA and AMPA drive in the models of the lamprey spinal CPG , 2003, Neurocomputing.

[77]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[78]  Dominique Martinez,et al.  An event-driven framework for the simulation of networks of spiking neurons , 2003, ESANN.

[79]  Germán Mato,et al.  On Numerical Simulations of Integrate-and-Fire Neural Networks , 1998, Neural Computation.

[80]  P. Kanold,et al.  Encoding the timing of inhibitory inputs. , 2005, Journal of neurophysiology.

[81]  Peter M. A. Sloot,et al.  Distributed Simulation with Cellular Automata: Architecture and Applications , 1999, SOFSEM.

[82]  Peter van Emde Boas,et al.  Design and implementation of an efficient priority queue , 1976, Mathematical systems theory.

[83]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[84]  Michael L. Hines,et al.  Parallel network simulations with NEURON , 2006, Journal of Computational Neuroscience.

[85]  H. Markram,et al.  Potential for multiple mechanisms, phenomena and algorithms for synaptic plasticity at single synapses , 1998, Neuropharmacology.

[86]  E Fransén,et al.  A model of cortical associative memory based on a horizontal network of connected columns. , 1998, Network.

[87]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[88]  Louis Tao,et al.  Efficient and Accurate Time-Stepping Schemes for Integrate-and-Fire Neuronal Networks , 2001, Journal of Computational Neuroscience.

[89]  James M. Bower,et al.  The Book of GENESIS , 1994, Springer New York.

[90]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[91]  Simon J Thorpe,et al.  SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons , 2003, Network.

[92]  Warren M. Grill,et al.  Temporal excitation properties of paresthesias evoked by thalamic microstimulation , 2005, Clinical Neurophysiology.

[93]  Mathilde Badoual,et al.  High discharge variability in neurons driven by current noise , 2005, Neurocomputing.

[94]  Michael L. Hines,et al.  Independent Variable Time-Step Integration of Individual Neurons for Network Simulations , 2005, Neural Computation.

[95]  E. Perez-Reyes,et al.  Functional Characterization and Neuronal Modeling of the Effects of Childhood Absence Epilepsy Variants of CACNA1H, a T-Type Calcium Channel , 2005, The Journal of Neuroscience.

[96]  S. Prescott,et al.  Integration Time in a Subset of Spinal Lamina I Neurons Is Lengthened by Sodium and Calcium Currents Acting Synergistically to Prolong Subthreshold Depolarization , 2005, The Journal of Neuroscience.

[97]  Gina G. Turrigiano,et al.  Homeostatic Synaptic Plasticity , 2008 .

[98]  M. Hirsch,et al.  Differential Equations, Dynamical Systems, and Linear Algebra , 1974 .

[99]  Idan Segev,et al.  Methods in neuronal modeling: From synapses to networks , 1989 .

[100]  R. Traub,et al.  Neuronal Networks of the Hippocampus , 1991 .

[101]  L. Finkel,et al.  NMDA/AMPA Ratio Impacts State Transitions and Entrainment to Oscillations in a Computational Model of the Nucleus Accumbens Medium Spiny Projection Neuron , 2005, The Journal of Neuroscience.

[102]  William W. Lytton.  Optimizing Synaptic Conductance Calculation for Network Simulations , 1996, Neural Computation.

[103]  Carol S. Woodward,et al.  Enabling New Flexibility in the SUNDIALS Suite of Nonlinear and Differential/Algebraic Equation Solvers , 2020, ACM Trans. Math. Softw..

[104]  Markus Diesmann,et al.  Exact Subthreshold Integration with Continuous Spike Times in Discrete-Time Neural Network Simulations , 2007, Neural Computation.

[105]  L. Abbott,et al.  Cortical Development and Remapping through Spike Timing-Dependent Plasticity , 2001, Neuron.

[106]  H. Markram,et al.  Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. , 1997, The Journal of physiology.

[107]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[108]  Netta Cohen,et al.  Real time computation: Zooming in on population codes , 2007, Biosyst..

[109]  Romain Brette,et al.  Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity , 2005 .

[110]  Alain Destexhe,et al.  Analytical Integrate-and-Fire Neuron Models with Conductance-Based Dynamics for Event-Driven Simulation Strategies , 2006, Neural Computation.

[111]  Carlo R. Laing,et al.  On the application of “equation-free modelling” to neural systems , 2006, Journal of Computational Neuroscience.

[112]  Christophe Bernard,et al.  Synaptic integration of NMDA and non-NMDA receptors in large neuronal network models solved by means of differential equations , 2004, Biological Cybernetics.

[113]  Örjan Ekeberg,et al.  Massively parallel simulation of brain-scale neuronal network models , 2005 .

[114]  Lloyd Watts,et al.  Event-Driven Simulation of Networks of Spiking Neurons , 1993, NIPS.

[115]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[116]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[117]  David Beeman,et al.  GENESIS Modeling Tutorial , 2005 .

[118]  Bard Ermentrout,et al.  Simulating, analyzing, and animating dynamical systems - a guide to XPPAUT for researchers and students , 2002, Software, environments, tools.

[119]  M. E. Galassi,et al.  GNU SCIENTI C LIBRARY REFERENCE MANUAL , 2005 .

[120]  U. Bhalla.  Signaling in small subcellular volumes. II. Stochastic and diffusion effects on synaptic network properties. , 2004, Biophysical journal.

[121]  Rich Drewes.  Modeling the brain with NCS and brainlab , 2005 .

[122]  Randy Brown,et al.  Calendar queues: a fast 0(1) priority queue implementation for the simulation event set problem , 1988, CACM.

[123]  M. Hines,et al.  Efficient computation of branched nerve equations. , 1984, International journal of bio-medical computing.

[124]  Philip H. Goodman,et al.  193 IN SILICO KNOCKIN/KNOCKOUT IN MODEL NEOCORTEX SUGGESTS ROLE OF Ca++-DEPENDENT K+ CHANNELS IN SPIKE-TIMING INFORMATION TRANSFER , 2005, Journal of Investigative Medicine.

[125]  M. Moffitt,et al.  Model-based analysis of cortical recording with silicon microelectrodes , 2005, Clinical Neurophysiology.

[126]  Ronan G. Reilly,et al.  Efficient event-driven simulation of spiking neural networks , 2002 .

[127]  Nabil H. Farhat,et al.  The double queue method: a numerical method for integrate-and-fire neuron networks , 2001, Neural Networks.

[128]  Terrence J. Sejnowski,et al.  Synthesis of models for excitable membranes, synaptic transmission and neuromodulation using a common kinetic formalism , 1994, Journal of Computational Neuroscience.

[129]  Michele Giugliano,et al.  Fast Calculation of Short-Term Depressing Synaptic Conductances , 1999, Neural Computation.

[130]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[131]  L. Arnold.  Stochastic Differential Equations: Theory and Applications , 1992 .

[132]  Sushil J. Louis,et al.  An Evolutionary Autonomous Agent with Visual Cortex and Recurrent Spiking Columnar Neural Network , 2004, GECCO.

[133]  James M. Bower,et al.  Rallpacks: a set of benchmarks for neuronal simulators , 1992, Trends in Neurosciences.

[134]  Brian W. Kernighan,et al.  The UNIX™ programming environment , 1979, Softw. Pract. Exp..

[135]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[136]  G. Ermentrout,et al.  Parabolic bursting in an excitable system coupled with a slow oscillation , 1986 .

[137]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[138]  Martin Rehn,et al.  Attractor dynamics in a modular network model of neocortex , 2006, Network.

[139]  D. Surmeier,et al.  Dendritic Excitability of Mouse Frontal Cortex Pyramidal Neurons Is Shaped by the Interaction among HCN, Kir2, and Kleak Channels , 2005, The Journal of Neuroscience.

[140]  Nicholas T. Carnevale,et al.  Expanding NEURON's Repertoire of Mechanisms with NMODL , 2000, Neural Computation.

[141]  M L Hines,et al.  Neuron: A Tool for Neuroscientists , 2001, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[142]  Marc-Oliver Gewaltig,et al.  NEST: An Environment for Neural Systems Simulations , 2003 .

[143]  H. Markram,et al.  Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. , 2000, Science.

[144]  Jonathan Bell,et al.  Introduction to Theoretical Neurobiology. Volume 1: Linear Cable Theory and Dendritic Structure. Volume 2: Nonlinear and Stochastic Theories (Henry C. Tuckwell) , 1990, SIAM Rev..