NeuroFlow: A General Purpose Spiking Neural Network Simulation Platform using Customizable Processors

NeuroFlow is a scalable spiking neural network simulation platform for off-the-shelf high performance computing systems using customizable hardware processors such as Field-Programmable Gate Arrays (FPGAs). Unlike multi-core processors and application-specific integrated circuits, the processor architecture of NeuroFlow can be redesigned and reconfigured to suit a particular simulation to deliver optimized performance, such as the degree of parallelism to employ. The compilation process supports using PyNN, a simulator-independent neural network description language, to configure the processor. NeuroFlow supports a number of commonly used current or conductance based neuronal models such as integrate-and-fire and Izhikevich models, and the spike-timing-dependent plasticity (STDP) rule for learning. A 6-FPGA system can simulate a network of up to ~600,000 neurons and can achieve a real-time performance of 400,000 neurons. Using one FPGA, NeuroFlow delivers a speedup of up to 33.6 times the speed of an 8-core processor, or 2.83 times the speed of GPU-based platforms. With high flexibility and throughput, NeuroFlow provides a viable environment for large-scale neural network simulation.

[1]  Simon R. Schultz,et al.  A parallel spiking neural network simulator , 2009, 2009 International Conference on Field-Programmable Technology.

[2]  Allan R. Jones,et al.  A mesoscale connectome of the mouse brain , 2014, Nature.

[3]  Allan R. Jones,et al.  An anatomically comprehensive atlas of the adult human brain transcriptome , 2012, Nature.

[4]  Michael W. Reimann,et al.  A Biophysically Detailed Model of Neocortical Local Field Potentials Predicts the Critical Role of Active Membrane Currents , 2013, Neuron.

[5]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[6]  Jason Cong,et al.  FPGA Simulation Engine for Customized Construction of Neural Microcircuit , 2013, 2013 IEEE 21st Annual International Symposium on Field-Programmable Custom Computing Machines.

[7]  Brent E. Nelson,et al.  HMFlow: Accelerating FPGA Compilation with Hard Macros for Rapid Prototyping , 2011, 2011 IEEE 19th Annual International Symposium on Field-Programmable Custom Computing Machines.

[8]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[9]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[10]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[11]  Tarek M. Taha,et al.  FPGA Implementation of Izhikevich Spiking Neural Networks for Character Recognition , 2009, 2009 International Conference on Reconfigurable Computing and FPGAs.

[12]  Sue L. Denham,et al.  Spatio-temporal pattern recognizers using spiking neurons and spike-timing-dependent plasticity , 2012, Front. Comput. Neurosci..

[13]  Johannes Schemmel,et al.  A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems , 2010, Biological Cybernetics.

[14]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[15]  R. W. Draft,et al.  Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system , 2007, Nature.

[16]  Steve B. Furber,et al.  The Deferred Event Model for Hardware-Oriented Spiking Neural Networks , 2008, ICONIP.

[17]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[18]  A. Cassidy,et al.  FPGA Based Silicon Spiking Neural Array , 2007, 2007 IEEE Biomedical Circuits and Systems Conference.

[19]  Allan R. Jones,et al.  A robust and high-throughput Cre reporting and characterization system for the whole mouse brain , 2009, Nature Neuroscience.

[20]  D. Johnston,et al.  Temporal synchrony and gamma to theta power conversion in the dendrites of CA1 pyramidal neurons , 2013, Nature Neuroscience.

[21]  Boris S. Gutkin,et al.  Passive Dendrites Enable Single Neurons to Compute Linearly Non-separable Functions , 2013, PLoS Comput. Biol..

[22]  André van Schaik,et al.  A neuromorphic implementation of multiple spike-timing synaptic plasticity rules for large-scale neural networks , 2015, Front. Neurosci..

[23]  Wayne Luk,et al.  FPGA Accelerated Simulation of Biologically Plausible Spiking Neural Networks , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[24]  Petr Lánský,et al.  Stochastic model neuron without resetting of dendritic potential: application to the olfactory system , 2004, Biological Cybernetics.

[25]  Wayne Luk,et al.  A Large-Scale Spiking Neural Network Accelerator for FPGA Systems , 2012, ICANN.

[26]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[27]  Nikil D. Dutt,et al.  An Efficient Simulation Environment for Modeling Large-Scale Cortical Processing , 2011, Front. Neuroinform..

[28]  Baktash Babadi,et al.  Intrinsic Stability of Temporally Shifted Spike-Timing Dependent Plasticity , 2010, PLoS Comput. Biol..

[29]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[30]  Tobias Becker,et al.  Modular dynamic reconfiguration in Virtex FPGAs , 2006 .

[31]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[32]  Wulfram Gerstner,et al.  Spiking Neuron Models: Formal spiking neuron models , 2002 .

[33]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[34]  Ammar Belatreche,et al.  Challenges for large-scale implementations of spiking neural networks on FPGAs , 2007, Neurocomputing.

[35]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[36]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[37]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[38]  Gregory Cohen,et al.  An FPGA Implementation of a Polychronous Spiking Neural Network with Delay Adaptation , 2013, Front. Neurosci..

[39]  Andrew S. Cassidy,et al.  Design of a one million neuron single FPGA neuromorphic system for real-time multimodal scene analysis , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[40]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[41]  Irene Yates,et al.  Cat Out of the Bag , 2017 .

[42]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[43]  Frederick C. Harris,et al.  A novel CPU/GPU simulation environment for large-scale biologically realistic neural modeling , 2013, Front. Neuroinform..

[44]  Karlheinz Meier,et al.  Introducing the Human Brain Project , 2011, FET.

[45]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[46]  Steve B. Furber,et al.  A General-Purpose Model Translation System for a Universal Neural Chip , 2010, ICONIP.

[47]  Robert H. Lee,et al.  An FPGA-based approach to high-speed simulation of conductance-based neuron models , 2007, Neuroinformatics.

[48]  Simon W. Moore,et al.  Bluehive - A field-programable custom computing machine for extreme-scale real-time neural network simulation , 2012, 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines.

[49]  Georgios Smaragdos,et al.  FPGA-based biophysically-meaningful modeling of olivocerebellar neurons , 2014, FPGA.

[50]  Kathie L. Olsen,et al.  Neurotech for Neuroscience: Unifying Concepts, Organizing Principles, and Emerging Tools , 2007, The Journal of Neuroscience.

[51]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[52]  Steve Furber,et al.  Real-time million-synapse simulation of rat barrel cortex , 2014, Front. Neurosci..

[53]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[54]  Mauricio Barahona,et al.  Sensory experience modifies spontaneous state dynamics in a large-scale barrel cortical model , 2012, Journal of Computational Neuroscience.

[55]  Murray Shanahan,et al.  Accelerated simulation of spiking neural networks using GPUs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[56]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[57]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model , 2009, Scholarpedia.

[58]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[59]  Luis A. Plana,et al.  A GALS Infrastructure for a Massively Parallel Multiprocessor , 2007, IEEE Design & Test of Computers.

[60]  Ian R. Wickersham,et al.  Cell type–specific genetic and optogenetic tools reveal hippocampal CA2 circuits , 2013 .

[61]  Ian R. Wickersham,et al.  Cell type-specific genetic and optogenetic tools reveal novel hippocampal CA2 circuits , 2013, Nature Neuroscience.

[62]  Lubica Benusková,et al.  STDP rule endowed with the BCM sliding threshold accounts for hippocampal heterosynaptic plasticity , 2007, Journal of Computational Neuroscience.