Virtues, Pitfalls, and Methodology of Neuronal Network Modeling and Simulations on Supercomputers

The number of neurons and synapses in biological brains is very large, on the order of millions and billions respectively even in small animals like insects and mice. By comparison most neuronal network models developed and simulated up to now have been tiny, comprising many orders of magnitude less neurons than their real counterpart, with an even more dramatic difference when it comes to the number of synapses. In this chapter we discuss why and when it may be important to work with large-scale, if not full-scale, neuronal network and brain models and to run simulations on supercomputers. We describe the state-of-the-art in large-scale neural simulation technology and methodology as well as ways to analyze and visualize output from such simulations. Finally we discuss the challenges and future trends in this field.

[1]  R. Buxton,et al.  Modeling the hemodynamic response to brain activation , 2004, NeuroImage.

[2]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[3]  Yun Wang,et al.  Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2-5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. , 2002, Cerebral cortex.

[4]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[5]  Moritz Helias,et al.  A General and Efficient Method for Incorporating Precise Spike Times in Globally Time-Driven Simulations , 2010, Front. Neuroinform..

[6]  F. Helmchen,et al.  In vivo calcium imaging of neural network function. , 2007, Physiology.

[7]  A. Lansner.  Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations , 2009, Trends in Neurosciences.

[8]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[9]  Michael L. Hines,et al.  Neuroinformatics Original Research Article Neuron and Python , 2022 .

[10]  Anders Lansner,et al.  Workshop report: 1st INCF Workshop on Large-scale Modeling of the Nervous System , 2007 .

[11]  M. Colonnier,et al.  The number of neurons in the different laminae of the binocular and monocular regions of area 17 in the cat , 1983, The Journal of comparative neurology.

[12]  Murray Shanahan,et al.  Accelerated simulation of spiking neural networks using GPUs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[13]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[14]  Örjan Ekeberg,et al.  Brain-scale simulation of the neocortex on the IBM Blue Gene/L supercomputer , 2008, IBM J. Res. Dev..

[15]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[16]  Michael L. Hines,et al.  Parallel network simulations with NEURON , 2006, Journal of Computational Neuroscience.

[17]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[18]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.

[19]  R. Traub,et al.  On the Mechanism of the γ → β Frequency Shift in Neuronal Oscillations Induced in Rat Hippocampal Slices by Tetanic Stimulation , 1999, The Journal of Neuroscience.

[20]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[21]  Germán Mato,et al.  Synchrony in Excitatory Neural Networks , 1995, Neural Computation.

[22]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[23]  Marc-Oliver Gewaltig,et al.  NEST: An Environment for Neural Systems Simulations , 2003 .

[24]  James G. King,et al.  A Component-Based Extension Framework for Large-Scale Parallel Simulations in NEURON , 2009, Front. Neuroinform..

[25]  Johannes Schemmel,et al.  Modeling Synaptic Plasticity within Networks of Highly Accelerated I&F Neurons , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[26]  Romain Brette,et al.  Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity , 2005 .

[27]  Terry Winograd,et al.  Breaking the complexity barrier again , 1973, SIGPLAN '73.

[28]  Michael L. Hines,et al.  NeuroML: A Language for Describing Data Driven Models of Neurons and Networks with a High Degree of Biological Detail , 2010, PLoS Comput. Biol..

[29]  Jürgen Kurths,et al.  Lectures in Supercomputational Neuroscience: Dynamics in Complex Brain Networks , 2007 .

[30]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[31]  Mark A. Ratner,et al.  Molecular electronics , 2005 .

[32]  Claude Bédard,et al.  Comparative power spectral analysis of simultaneous elecroencephalographic and magnetoencephalographic recordings in humans suggests non-resistive extracellular media , 2010, Journal of Computational Neuroscience.

[33]  Markus Diesmann,et al.  Frontiers in Computational Neuroscience Enabling Functional Neural Circuit Simulations with Distributed Computing of Neuromodulated Plasticity , 2022 .

[34]  Markus Diesmann,et al.  Simplicity and Efficiency of Integrate-and-Fire Neuron Models , 2009, Neural Computation.

[35]  O. Garaschuk,et al.  Wide-field and two-photon imaging of brain activity with voltage- and calcium-sensitive dyes , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[36]  Ammar Belatreche,et al.  Challenges for large-scale implementations of spiking neural networks on FPGAs , 2007, Neurocomputing.

[37]  G. Ermentrout,et al.  Parabolic bursting in an excitable system coupled with a slow oscillation , 1986 .

[38]  Martin Rehn,et al.  Attractor dynamics in a modular network model of neocortex , 2006, Network.

[39]  Mikael Djurfeldt.  The Connection-set Algebra—A Novel Formalism for the Representation of Connectivity Structure in Neuronal Network Models , 2012, Neuroinformatics.

[40]  Örjan Ekeberg,et al.  Run-Time Interoperability Between Neuronal Network Simulators Based on the MUSIC Framework , 2010, Neuroinformatics.

[41]  Simon R. Schultz,et al.  A parallel spiking neural network simulator , 2009, 2009 International Conference on Field-Programmable Technology.

[42]  Mark Zwolinski,et al.  A communication infrastructure for a million processor machine , 2010, CF '10.

[43]  Örjan Ekeberg,et al.  Large Neural Network Simulations on Multiple Hardware Platforms , 1997, Journal of Computational Neuroscience.

[44]  A. Lansner,et al.  Modelling Hebbian cell assemblies comprised of cortical neurons , 1992 .

[45]  Michael A. Arbib,et al.  Computing the brain : a guide to neuroinformatics , 2001 .

[46]  Markus Diesmann,et al.  Exact Subthreshold Integration with Continuous Spike Times in Discrete-Time Neural Network Simulations , 2007, Neural Computation.

[47]  Gaute T. Einevoll,et al.  Intrinsic dendritic filtering gives low-pass power spectra of local field potentials , 2010, Journal of Computational Neuroscience.

[48]  Anders Lansner,et al.  Theta and Gamma Power Increases and Alpha/Beta Power Decreases with Memory Load in an Attractor Network Model , 2011, Journal of Cognitive Neuroscience.

[49]  Markus Diesmann,et al.  Programmable Logic Construction Kits for Hyper-Real-Time Neuronal Modeling , 2006, Neural Computation.

[50]  Michael L. Hines,et al.  Fully implicit parallel simulation of single neurons , 2008, Journal of Computational Neuroscience.

[51]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[52]  Andreas Mayr,et al.  CrossNets: High‐Performance Neuromorphic Architectures for CMOL Circuits , 2003, Annals of the New York Academy of Sciences.

[53]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[54]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[55]  Nelson J. Trujillo-Barreto,et al.  Biophysical model for integrating neuronal activity, EEG, fMRI and metabolism , 2008, NeuroImage.

[56]  Sonja Grün,et al.  Data-driven significance estimation for precise spike correlation. , 2009, Journal of neurophysiology.

[57]  Steve B. Furber,et al.  Algorithm and software for simulation of spiking neural networks on the multi-chip SpiNNaker system , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[58]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[59]  G. B. Ermentrout,et al.  Large Amplitude Stationary Waves in an Excitable Lateral-Inhibitory Medium , 1984 .

[60]  Paul R. Schrater,et al.  Within- and Cross-Modal Distance Information Disambiguate Visual Size-Change Perception , 2010, PLoS Comput. Biol..

[61]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[62]  Markus Diesmann,et al.  Maintaining Causality in Discrete Time Neuronal Network Simulations , 2007 .

[63]  N. Logothetis,et al.  From Neurons to Circuits: Linear Estimation of Local Field Potentials , 2009, The Journal of Neuroscience.

[64]  Ronald A. J. van Elburg,et al.  Generalization of the Event-Based Carnevale-Hines Integration Scheme for Integrate-and-Fire Models , 2009, Neural Computation.

[65]  F. Chavane,et al.  Voltage-sensitive dye imaging: Technique review and models , 2010, Journal of Physiology-Paris.

[66]  Markus Diesmann,et al.  The cell-type specific connectivity of the local cortical network explains prominent features of neuronal activity , 2011, 1106.5678.

[67]  C. Mead,et al.  Neuromorphic analogue VLSI. , 1995, Annual review of neuroscience.

[68]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[69]  Ronald J. MacGregor,et al.  Neural and brain modeling , 1987 .

[70]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[71]  P. Somogyi,et al.  Quantitative distribution of GABA-immunoreactive neurons in the visual cortex (area 17) of the cat , 2004, Experimental Brain Research.

[72]  D. Hansel,et al.  How Spike Generation Mechanisms Determine the Neuronal Response to Fluctuating Inputs , 2003, The Journal of Neuroscience.

[73]  Anders Lansner,et al.  Bistable, Irregular Firing and Population Oscillations in a Modular Attractor Memory Network , 2010, PLoS Comput. Biol..

[74]  Wayne Luk,et al.  FPGA Accelerated Simulation of Biologically Plausible Spiking Neural Networks , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[75]  Benjamin Schrauwen,et al.  Fast and Exact Simulation Methods Applied on a Broad Range of Neuron Models , 2010, Neural Computation.

[76]  Shigeru Shinomoto,et al.  Made-to-Order Spiking Neuron Model Equipped with a Multi-Timescale Adaptive Threshold , 2009, Front. Comput. Neurosci..

[77]  Marc-Oliver Gewaltig,et al.  Towards Reproducible Descriptions of Neuronal Network Models , 2009, PLoS Comput. Biol..

[78]  Hamid Soltanian-Zadeh,et al.  Multi-area neural mass modeling of EEG and MEG signals , 2010, NeuroImage.

[79]  Erik De Schutter.  Why Are Computational Neuroscience and Systems Biology So Separate? , 2008, PLoS Comput. Biol..

[80]  Dan D. Stettler,et al.  Representations of Odor in the Piriform Cortex , 2009, Neuron.

[81]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[82]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[83]  Claude Bédard,et al.  Comparative power spectral analysis of simultaneous elecroencephalographic and magnetoencephalographic recordings in humans suggests non-resistive extracellular media , 2010, Journal of Computational Neuroscience.

[84]  Steve B. Furber,et al.  Modeling Spiking Neural Networks on SpiNNaker , 2010, Computing in Science & Engineering.

[85]  Giacomo Indiveri,et al.  Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition , 2009, Cognitive Computation.

[86]  Nicolas Brunel,et al.  Lapicque’s 1907 paper: from frogs to integrate-and-fire , 2007, Biological Cybernetics.

[87]  R. Kötter,et al.  Connecting Mean Field Models of Neural Activity to EEG and fMRI Data , 2010, Brain Topography.

[88]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[89]  D. Noble.  The music of life : biology beyond genes , 2008 .