From von Neumann Architecture and Atanasoffs ABC to Neuro-Morphic Computation and Kasabov’s NeuCube: Principles and Implementations

During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from http://www.kedri.aut.ac.nz/neucube/.

[1]  R. Schaller,et al.  Moore's law: past, present and future , 1997 .

[2]  Maryam Gholami Doborjeh,et al.  Classification and segmentation of fMRI Spatio-Temporal Brain Data with a NeuCube evolving Spiking Neural Network model , 2014, 2014 IEEE Symposium on Evolving and Autonomous Learning Systems (EALS).

[3]  Tobi Delbrück,et al.  Fast sensory motor control based on event-based hybrid neuromorphic-procedural system , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[4]  Stefan Marks,et al.  Immersive visualisation of 3-dimensional neural network structures , 2015 .

[5]  Rich Drewes Brainlab: a toolkit to aid in the design, simulation, and analysis of spiking neural networks with the NCS environment , 2005 .

[6]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[7]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[8]  Nikola K. Kasabov,et al.  From von neumann, John Atanasoff and ABC to Neuromorphic computation and the NeuCube spatio-temporal data machine , 2016, 2016 IEEE 8th International Conference on Intelligent Systems (IS).

[9]  Chris Toumey Less is Moore. , 2016, Nature nanotechnology.

[10]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[11]  Stefano Fusi Spike-driven Synaptic Plasticity for Learning Correlated Patterns of Mean Firing Rates , 2003, Reviews in the neurosciences.

[12]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Ivan K. Schuller,et al.  Neuromorphic Computing – From Materials Research to Systems Architecture Roundtable , 2015 .

[15]  Moritz Helias,et al.  Neuroinformatics Original Research Article Pynest: a Convenient Interface to the Nest Simulator , 2022 .

[16]  Nikola K. Kasabov,et al.  NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data , 2014, Neural Networks.

[17]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[18]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[19]  Stefan Schliebs,et al.  Span: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns , 2012, Int. J. Neural Syst..

[20]  Dan F. M. Goodman Code Generation: A Strategy for Neural Network Simulators , 2010, Neuroinformatics.

[21]  Maryam Gholami Doborjeh,et al.  Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications , 2016, Neural Networks.

[22]  Nikola K. Kasabov,et al.  Integrated optimisation method for personalised modelling and case studies for medical decision support , 2010, Int. J. Funct. Informatics Pers. Medicine.

[23]  Jeremy Hsu,et al.  IBM's new brain [News] , 2014 .

[24]  Henry Markram,et al.  Computer models and analysis tools for neural microcircuits , 2003 .

[25]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[26]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[27]  Brian Randell,et al.  The origins of digital computers: Selected papers , 1975 .

[28]  Karthikeyan Sankaralingam,et al.  Dark silicon and the end of multicore scaling , 2011, 2011 38th Annual International Symposium on Computer Architecture (ISCA).

[29]  Nikola Kasabov,et al.  Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. , 2013, Neural networks : the official journal of the International Neural Network Society.

[30]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[31]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[32]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[33]  Maryam Gholami Doborjeh,et al.  Personalised modelling on integrated clinical and EEG Spatio-Temporal Brain Data in the NeuCube Spiking Neural Network system , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[34]  Giacomo Indiveri,et al.  NeuCube Neuromorphic Framework for Spatio-temporal Brain Data and Its Python Implementation , 2013, ICONIP.

[35]  Romain Brette,et al.  The Brian Simulator , 2009, Front. Neurosci..

[36]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[37]  Giacomo Indiveri,et al.  Memory and Information Processing in Neuromorphic Systems , 2015, Proceedings of the IEEE.

[38]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[39]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[40]  Enrico Macii,et al.  The Human Brain Project and neuromorphic computing. , 2013, Functional neurology.

[41]  Stefan Bruckner,et al.  BrainGazer - Visual Queries for Neurobiology Research , 2009, IEEE Transactions on Visualization and Computer Graphics.

[42]  Klaus Schuch,et al.  PCSIM: A Parallel Simulation Environment for Neural Circuits Fully Integrated with Python , 2008, Frontiers Neuroinformatics.

[43]  Thorsten Meinl,et al.  KNIME: The Konstanz Information Miner , 2007, GfKl.

[44]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[45]  Sheng-Chuan Wang,et al.  The Neuron Navigator: Exploring the information pathway through the neural maze , 2011, 2011 IEEE Pacific Visualization Symposium.

[46]  Dimitri Perrin,et al.  Complexity and high-end computing in biology and medicine. , 2011, Advances in Experimental Medicine and Biology.

[47]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[48]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[49]  Nikola Kasabov,et al.  Framework for Knowledge Driven Optimisation Based Data Encoding for Brain Data Modelling Using Spiking Neural Network Architecture , 2015 .

[50]  Stefan Marks Immersive visualisation of 3-dimensional spiking neural networks , 2017, Evol. Syst..

[51]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[52]  Nikola K. Kasabov,et al.  Analysis of connectivity in NeuCube spiking neural network models trained on EEG data for the understanding of functional changes in the brain: A case study on opiate dependence treatment , 2015, Neural Networks.

[53]  Markus Diesmann,et al.  Towards the Visualization of Spiking Neurons in Virtual Reality , 2011, MMVR.

[54]  Blaz Zupan,et al.  Orange: From Experimental Machine Learning to Interactive Data Mining , 2004, PKDD.

[55]  David Berlinski,et al.  The Advent of the Algorithm: The 300-Year Journey from an Idea to the Computer , 2000 .

[56]  S. Furber,et al.  To build a brain , 2012, IEEE Spectrum.