A Low Power Trainable Neuromorphic Integrated Circuit That Is Tolerant to Device Mismatch

Random device mismatch that arises as a result of scaling of the CMOS (complementary metal-oxide semiconductor) technology into the deep submicrometer regime degrades the accuracy of analog circuits. Methods to combat this increase the complexity of design. We have developed a novel neuromorphic system called a trainable analog block (TAB), which exploits device mismatch as a means for random projections of the input to a higher dimensional space. The TAB framework is inspired by the principles of neural population coding operating in the biological nervous system. Three neuronal layers, namely input, hidden, and output, constitute the TAB framework, with the number of hidden layer neurons far exceeding the input layer neurons. Here, we present measurement results of the first prototype TAB chip built using a 65 nm process technology and show its learning capability for various regression tasks. Our TAB chip is tolerant to inherent randomness and variability arising due to the fabrication process. Additionally, we characterize each neuron and discuss the statistical variability of its tuning curve that arises due to random device mismatch, a desirable property for the learning capability of the TAB. We also discuss the effect of the number of hidden neurons and the resolution of output weights on the accuracy of the learning capability of the TAB. We show that the TAB is a low power system-the power dissipation in the TAB with 456 neuron blocks is 1.38 μW.

[1]  Arindam Basu,et al.  A 128 channel 290 GMACs/W machine learning based co-processor for intention decoding in brain machine interfaces , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[2]  Tobi Delbrück,et al.  Bias Current Generators with Wide Dynamic Range , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[3]  L. Abbott,et al.  Random Convergence of Olfactory Inputs in the Drosophila Mushroom Body , 2013, Nature.

[4]  Jochen J. Steil,et al.  Device mismatch in a neuromorphic system implements random features for regression , 2015, 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[5]  S. Himavathi,et al.  Feedforward Neural Network Implementation in FPGA Using Layer Multiplexing for Effective Resource Utilization , 2007, IEEE Transactions on Neural Networks.

[6]  Giacomo Indiveri,et al.  Mapping arbitrary mathematical functions and dynamical systems to neuromorphic VLSI circuits for spike-based neural computation , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[7]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[8]  Andrew Marshall.  Mismatch and Noise in Modern IC Processes , 2009, Mismatch and Noise in Modern IC Processes.

[9]  J. Bacon,et al.  Receptive fields of cricket giant interneurones are related to their dendritic structure. , 1984, The Journal of physiology.

[10]  M. A. Jabri,et al.  A low power trainable analogue neural network classifier chip , 1993, Proceedings of IEEE Custom Integrated Circuits Conference - CICC '93.

[11]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[12]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[13]  L. Abbott,et al.  Theoretical Neuroscience Rising , 2008, Neuron.

[14]  Y. Takefuji,et al.  Functional-link net computing: theory, system architecture, and functionalities , 1992, Computer.

[15]  Emilio Salinas,et al.  Vector reconstruction from firing rates , 1994, Journal of Computational Neuroscience.

[16]  Valentin Dragoi,et al.  Efficient coding in heterogeneous neuronal populations , 2008, Proceedings of the National Academy of Sciences.

[17]  Y. Wang.  Analog CMOS implementation of backward error propagation , 1993, IEEE International Conference on Neural Networks.

[18]  Gregory Cohen,et al.  Synthesis of neural networks for spatio-temporal spike pattern recognition and processing , 2013, Front. Neurosci..

[19]  Terrence C. Stewart,et al.  Large-Scale Synthesis of Functional Spiking Neural Circuits , 2014, Proceedings of the IEEE.

[20]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[21]  Eric A. Vittoz,et al.  Weak inversion for ultra low-power and very low-voltage circuits , 2009, 2009 IEEE Asian Solid-State Circuits Conference.

[22]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[23]  D. G. Albrecht,et al.  Striate cortex of monkey and cat: contrast response function. , 1982, Journal of neurophysiology.

[24]  T. Sejnowski.  Neural populations revealed , 1988, Nature.

[25]  Alexander S. Ecker,et al.  The effect of noise correlations in populations of diversely tuned neurons , 2011 .

[26]  E I Knudsen,et al.  A neural map of auditory space in the owl. , 1978, Science.

[27]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[28]  Alan F. Murray,et al.  Spike timing dependent plasticity (STDP) can ameliorate process variations in neuromorphic VLSI , 2005, IEEE Transactions on Neural Networks.

[29]  J. V. Gisbergen,et al.  Collicular ensemble coding of saccades based on vector summation , 1987, Neuroscience.

[30]  André van Schaik,et al.  Stochastic Electronics: A Neuro-Inspired Design Paradigm for Integrated Circuits , 2014, Proceedings of the IEEE.

[31]  Chris Eliasmith,et al.  Neural Engineering (Computational Neuroscience Series): Computational, Representation, and Dynamics in Neurobiological Systems , 2002 .

[32]  Michel Steyaert,et al.  Analog VLSI implementation of Neural Networks , 1992 .

[33]  André van Schaik,et al.  Neuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[34]  Chan-Wang Park,et al.  FPGA-implementation of high-speed MLP neural network , 2011, 2011 18th IEEE International Conference on Electronics, Circuits, and Systems.

[35]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[36]  P.R. Kinget.  Device mismatch and tradeoffs in the design of analog circuits , 2005, IEEE Journal of Solid-State Circuits.

[37]  N Suga,et al.  Target range-sensitive neurons in the auditory cortex of the mustache bat. , 1979, Science.

[38]  A. Thakoor,et al.  Design of parallel hardware neural network systems from custom analog VLSI 'building block' chips , 1989, International 1989 Joint Conference on Neural Networks.

[39]  Hongming Zhou,et al.  Silicon spiking neurons for hardware implementation of extreme learning machines , 2013, Neurocomputing.

[40]  André van Schaik,et al.  An Online Learning Algorithm for Neuromorphic Hardware Implementation , 2015, ArXiv.

[41]  Jagath C. Rajapakse,et al.  FPGA Implementations of Neural Networks , 2006 .

[42]  Ray Andraka,et al.  A survey of CORDIC algorithms for FPGA based computers , 1998, FPGA '98.

[43]  André van Schaik,et al.  Learning the pseudoinverse solution to network weights , 2012, Neural Networks.

[44]  André van Schaik,et al.  A neuromorphic hardware framework based on population coding , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[45]  Florham Park.  Parameterized High Throughput Function Evaluation for FPGAs , .

[46]  Francisco José Ballester-Merelo,et al.  Artificial neural network implementation on a single FPGA of a pipelined on-line backpropagation , 2000, Proceedings 13th International Symposium on System Synthesis.

[47]  Dhireesha Kudithipudi,et al.  Neuromemristive Extreme Learning Machines for Pattern Classification , 2014, 2014 IEEE Computer Society Annual Symposium on VLSI.

[48]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[49]  Andrew B. Kahng,et al.  Design Challenges at 65nm and Beyond , 2007, 2007 Design, Automation & Test in Europe Conference & Exhibition.

[50]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.