Compass: A scalable simulator for an architecture for cognitive computing

Inspired by the function, power, and volume of the organic brain, we are developing TrueNorth, a novel modular, non-von Neumann, ultra-low power, compact architecture. TrueNorth consists of a scalable network of neurosynaptic cores, with each core containing neurons, dendrites, synapses, and axons. To set sail for TrueNorth, we developed Compass, a multi-threaded, massively parallel functional simulator and a parallel compiler that maps a network of long-distance pathways in the macaque monkey brain to TrueNorth. We demonstrate near-perfect weak scaling on a 16 rack IBM® Blue Gene®/Q (262144 CPUs, 256 TB memory), achieving an unprecedented scale of 256 million neurosynaptic cores containing 65 billion neurons and 16 trillion synapses running only 388x slower than real time with an average spiking rate of 8.1 Hz. By using emerging PGAS communication primitives, we also demonstrate 2x better real-time performance over MPI primitives on a 4 rack Blue Gene/P (16384 CPUs, 16 TB memory).

[1]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[2]  E. D’Angelo.  The human brain project. , 2012, Functional neurology.

[3]  I. Olkin,et al.  Scaling of matrices to achieve specified row and column sums , 1968 .

[4]  Romain Brette,et al.  Vectorized Algorithms for Spiking Neural Network Simulation , 2011, Neural Computation.

[5]  Philip Heidelberger,et al.  The IBM Blue Gene/Q interconnection network and message unit , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[6]  Rolf Kötter,et al.  Online retrieval, processing, and visualization of primate connectivity data from the CoCoMac Database , 2007, Neuroinformatics.

[7]  Andrew S. Cassidy,et al.  Building block of a programmable neuromorphic substrate: A digital neurosynaptic core , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[8]  Christof Koch,et al.  Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series) , 1998 .

[9]  John W. Backus,et al.  Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs , 1978, CACM.

[10]  Torsten Hoefler,et al.  Scalable communication protocols for dynamic sparse data exchange , 2010, PPoPP '10.

[11]  Alan Peters,et al.  Association and Auditory Cortices , 1985, Cerebral Cortex.

[12]  Dharmendra S. Modha,et al.  A Digital Neurosynaptic Core Using Event-Driven QDI Circuits , 2012, 2012 IEEE 18th International Symposium on Asynchronous Circuits and Systems.

[13]  Dharmendra S. Modha,et al.  A Conceptual Cortical Surface Atlas , 2009, PloS one.

[14]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[15]  D. B. Leitch,et al.  Neuron densities vary across and within cortical areas in primates , 2010, Proceedings of the National Academy of Sciences.

[16]  A. Toga,et al.  The Rhesus Monkey Brain in Stereotaxic Coordinates , 1999 .

[17]  Geoffrey W. Burr,et al.  Nanoscale electronic synapses using phase change devices , 2013, JETC.

[18]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[19]  R. E. Passingham,et al.  Cerebral cortex, vol. 4, association and auditory cortices A. Peters andE. G. Jones (eds). Plenum Press, New York (1985). 359 pp. £52.25 , 1987, Neuroscience.

[20]  Nikil D. Dutt,et al.  Efficient simulation of large-scale Spiking Neural Networks using CUDA graphics processors , 2009, 2009 International Joint Conference on Neural Networks.

[21]  Michael Gschwind,et al.  The IBM Blue Gene/Q Compute Chip , 2012, IEEE Micro.

[22]  V. Mountcastle.  The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[23]  Philip Heidelberger,et al.  The deep computing messaging framework: generalized scalable message passing on the blue gene/P supercomputer , 2008, ICS '08.

[24]  Richard Sinkhorn,et al.  Concerning nonnegative matrices and doubly stochastic matrices , 1967 .

[25]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[26]  Dan Bonachea.  GASNet Specification, v1.1 , 2002 .

[27]  Jean,et al.  The Computer and the Brain , 1989, Annals of the History of Computing.

[28]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[29]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[30]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[31]  D. Modha,et al.  Network architecture of the long-distance pathways in the macaque brain , 2010, Proceedings of the National Academy of Sciences.

[32]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[33]  Brian A. Wandell,et al.  Think Global, Act Local; Projectome Estimation with BlueMatter , 2009, MICCAI.

[34]  Örjan Ekeberg,et al.  Brain-scale simulation of the neocortex on the IBM Blue Gene/L supercomputer , 2008, IBM J. Res. Dev..

[35]  Philip A. Knight,et al.  The Sinkhorn-Knopp Algorithm: Convergence and Applications , 2008, SIAM J. Matrix Anal. Appl..

[36]  Dharmendra S. Modha,et al.  Anatomy of a cortical simulator , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[37]  Ben Goertzel,et al.  A world survey of artificial brain projects, Part I: Large-scale brain simulations , 2010, Neurocomputing.

[38]  Katherine A. Yelick,et al.  Scaling communication-intensive applications on BlueGene/P using one-sided communication and overlap , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[39]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[40]  V. Mountcastle.  Perceptual Neuroscience: The Cerebral Cortex , 1998 .

[41]  M. Mitchell Waldrop,et al.  Computer modelling: Brain in a box , 2012, Nature.