Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing

The availability of efficient and reliable simulation tools is one of the mission-critical technologies in the fast-moving field of computational neuroscience. Research indicates that higher brain functions emerge from large and complex cortical networks and their interactions. The large number of elements (neurons) combined with the high connectivity (synapses) of the biological network and the specific type of interactions impose severe constraints on the explorable system size that previously have been hard to overcome. Here we present a collection of new techniques combined to a coherent simulation tool removing the fundamental obstacle in the computational study of biological neural networks: the enormous number of synaptic contacts per neuron. Distributing an individual simulation over multiple computers enables the investigation of networks orders of magnitude larger than previously possible. The software scales excellently on a wide range of tested hardware, so it can be used in an interactive and iterative fashion for the development of ideas, and results can be produced quickly even for very large networks. In con-trast to earlier approaches, a wide class of neuron models and synaptic dynamics can be represented.

[1]  W. J. Nowack Methods in Neuronal Modeling , 1991, Neurology.

[2]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[3]  Louis Tao,et al.  Efficient and Accurate Time-Stepping Schemes for Integrate-and-Fire Neuronal Networks , 2001, Journal of Computational Neuroscience.

[4]  James M. Bower,et al.  The Book of GENESIS , 1994, Springer New York.

[5]  Stefan Rotter,et al.  Higher-Order Statistics of Input Ensembles and the Response of Simple Model Neurons , 2003, Neural Computation.

[6]  Markus Diesmann,et al.  Exact Subthreshold Integration with Continuous Spike Times in Discrete-Time Neural Network Simulations , 2007, Neural Computation.

[7]  Bernard Harris,et al.  Graph theory and its applications , 1970 .

[8]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[9]  Michael L. Hines,et al.  Parallel network simulations with NEURON , 2006, Journal of Computational Neuroscience.

[10]  Nabil H. Farhat,et al.  The double queue method: a numerical method for integrate-and-fire neuron networks , 2001, Neural Networks.

[11]  Roland Heim,et al.  Theoretical Approaches to Complex Systems , 1978 .

[12]  David R. Butenhof Programming with POSIX threads , 1993 .

[13]  A. Destexhe,et al.  The high-conductance state of neocortical neurons in vivo , 2003, Nature Reviews Neuroscience.

[14]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[15]  Christophe Bernard,et al.  Synaptic integration of NMDA and non-NMDA receptors in large neuronal network models solved by means of differential equations , 2004, Biological Cybernetics.

[16]  Markus Diesmann,et al.  Consequences of realistic network size on the stability of embedded synfire chains , 2004, Neurocomputing.

[17]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[18]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[19]  Simon J Thorpe,et al.  SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons , 2003, Network.

[20]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[21]  J. Deuchars,et al.  Temporal and spatial properties of local circuits in neocortex , 1994, Trends in Neurosciences.

[22]  Cho-Li Wang,et al.  Efficient Scheduling of Complete Exchange on Clusters , 2000 .

[23]  Bjarne Stroustrup,et al.  C++ Programming Language , 1986, IEEE Softw..

[24]  Germán Mato,et al.  On Numerical Simulations of Integrate-and-Fire Neural Networks , 1998, Neural Computation.

[25]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[26]  Idan Segev,et al.  Methods in Neuronal Modeling , 1988 .

[27]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[28]  Bard Ermentrout,et al.  Simulating, analyzing, and animating dynamical systems - a guide to XPPAUT for researchers and students , 2002, Software, environments, tools.

[29]  M. E. Galassi,et al.  GNU SCIENTI C LIBRARY REFERENCE MANUAL , 2005 .

[30]  Christof Koch,et al.  Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series) , 1998 .

[31]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[32]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[33]  D. Georgescauld Local Cortical Circuits, An Electrophysiological Study , 1983 .

[34]  M K Habib,et al.  Dynamics of neuronal firing correlation: modulation of "effective connectivity". , 1989, Journal of neurophysiology.

[35]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[36]  Bjarne Stroustrup,et al.  The C++ programming language (3. ed.) , 1997 .

[37]  L. Garey Cortex: Statistics and Geometry of Neuronal Connectivity, 2nd edn. By V. BRAITENBERG and A. SCHÜZ. (Pp. xiii+249; 90 figures; ISBN 3 540 63816 4). Berlin: Springer. 1998. , 1999 .

[38]  Wolf Singer,et al.  Time as coding space? , 1999, Current Opinion in Neurobiology.

[39]  Rolf Backofen,et al.  A bottom-up approach , 2016 .

[40]  Peter S. Pacheco Parallel programming with MPI , 1996 .

[41]  Idan Segev,et al.  Methods in neuronal modeling: From synapses to networks , 1989 .

[42]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[43]  Marc-Oliver Gewaltig,et al.  NEST: An Environment for Neural Systems Simulations , 2003 .

[44]  Moshe Abeles,et al.  On Embedding Synfire Chains in a Balanced Network , 2003, Neural Computation.

[45]  A. Hawkes Point Spectra of Some Mutually Exciting Point Processes , 1971 .

[46]  Stefan Rotter,et al.  State space analysis of synchronous spiking in cortical neural networks , 2001, Neurocomputing.

[47]  Frances S. Chance,et al.  Gain Modulation from Background Synaptic Input , 2002, Neuron.

[48]  Vicenç Gómez,et al.  Event modeling of message interchange in stochastic neural ensembles , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[49]  Markus Diesmann,et al.  Activity dynamics and propagation of synchronous spiking in locally connected random networks , 2003, Biological Cybernetics.

[50]  Romain Brette,et al.  Exact Simulation of Integrate-and-Fire Models with Synaptic Conductances , 2006, Neural Computation.

[51]  Donald E. Knuth The Art of Computer Programming 2 / Seminumerical Algorithms , 1971 .

[52]  Marc-Oliver Gewaltig,et al.  SYNOD: An Environment for Neural Systems Simulations Language Interface and Tutorial , 2007 .

[53]  A. Aertsen,et al.  Neuronal assemblies , 1989, IEEE Transactions on Biomedical Engineering.

[54]  B. Sakmann,et al.  Dendritic mechanisms underlying the coupling of the dendritic with the axonal action potential initiation zone of adult rat layer 5 pyramidal neurons , 2001, The Journal of physiology.

[55]  John Beidler,et al.  Data Structures and Algorithms , 1996, Wiley Encyclopedia of Computer Science and Engineering.

[56]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[57]  Prof. Dr. Dr. Valentino Braitenberg,et al.  Cortex: Statistics and Geometry of Neuronal Connectivity , 1998, Springer Berlin Heidelberg.

[58]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[59]  T. Sejnowski,et al.  Impact of Correlated Synaptic Input on Output Firing Rate and Variability in Simple Neuronal Models , 2000, The Journal of Neuroscience.

[60]  Geehyuk Lee,et al.  Erratum to: The double queue method: A numerical method for integrate-and-fire neuron networks [Neural Networks 14(6-7) 921-932] , 2002 .

[61]  A. Aertsen,et al.  Neuronal Integration of Synaptic Input in the Fluctuation-Driven Regime , 2004, The Journal of Neuroscience.

[62]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[63]  C. von der Malsburg,et al.  Am I Thinking Assemblies , 1986 .

[64]  Bjarne Stroustrup,et al.  The Design and Evolution of C , 1994 .

[65]  W. Singer Synchronization of cortical activity and its putative role in information processing and learning. , 1993, Annual review of physiology.

[66]  M. Abeles Local Cortical Circuits: An Electrophysiological Study , 1982 .

[67]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[68]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[69]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[70]  C. Nicholson Electric current flow in excitable cells J. J. B. Jack, D. Noble &R. W. Tsien Clarendon Press, Oxford (1975). 502 pp., £18.00 , 1976, Neuroscience.

[71]  Michele Giugliano,et al.  Event-Driven Simulation of Spiking Neurons with Stochastic Dynamics , 2003, Neural Computation.

[72]  Michael Allen,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers , 1998 .