Neural Dynamics in Reconfigurable Silicon

A neuromorphic analog chip is presented that is capable of implementing massively parallel neural computations while retaining the programmability of digital systems. We show measurements from neurons with Hopf bifurcations and integrate and fire neurons, excitatory and inhibitory synapses, passive dendrite cables, coupled spiking neurons, and central pattern generators implemented on the chip. This chip provides a platform for not only simulating detailed neuron dynamics but also uses the same to interface with actual cells in applications such as a dynamic clamp. There are 28 computational analog blocks (CAB), each consisting of ion channels with tunable parameters, synapses, winner-take-all elements, current sources, transconductance amplifiers, and capacitors. There are four other CABs which have programmable bias generators. The programmability is achieved using floating gate transistors with on-chip programming control. The switch matrix for interconnecting the components in CABs also consists of floating-gate transistors. Emphasis is placed on replicating the detailed dynamics of computational neural models. Massive computational area efficiency is obtained by using the reconfigurable interconnect as synaptic weights, resulting in more than 50 000 possible 9-b accurate synapses in 9 mm2.

[1]  Vittorio Dante,et al.  A VLSI recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory , 2003, IEEE Trans. Neural Networks.

[2]  Michiel Steyaert,et al.  A 10-bit 1-GSample/s Nyquist current-steering CMOS D/A converter , 2001 .

[3]  Arindam Basu,et al.  A Fully Integrated Architecture for Fast Programming of Floating Gates , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[4]  Juha Kostamovaara,et al.  A current-mode gain-control scheme with constant bandwidth and propagation delay for a transimpedance preamplifier , 1999 .

[5]  E.K.F. Lee,et al.  A CMOS field-programmable analog array , 1991 .

[6]  L. D. Tzeng,et al.  A 1.06 Gb/s -31 dBm to 0 dBm BiCMOS optical preamplifier featuring adaptive transimpedance , 1995, Proceedings ISSCC '95 - International Solid-State Circuits Conference.

[7]  G.E.R. Cowan,et al.  A VLSI analog computer/digital computer accelerator , 2006, IEEE Journal of Solid-State Circuits.

[8]  István Petrás,et al.  A bio-inspired two-layer mixed-signal flexible programmable chip for early vision , 2003, IEEE Trans. Neural Networks.

[9]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[10]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[11]  Tyson S. Hall,et al.  Automatic rapid programming of large arrays of floating-gate elements , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[12]  Giacomo Indiveri,et al.  A current-mode conductance-based silicon neuron for address-event neuromorphic systems , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[13]  Bradley A. Minch A low-voltage MOS cascode bias circuit for all current levels , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[14]  Bernabe Linares-Barranco,et al.  A CMOS Implementation of Fitzhugh-Nagumo Neuron Model , 1990, ESSCIRC '90: Sixteenth European Solid-State Circuits Conference.

[15]  M. J. Hayes A nonlinear optical preamplifier for sensing applications , 2002 .

[16]  S. Le Masson,et al.  Analog circuits for modeling biological neural networks: design and applications , 1999, IEEE Transactions on Biomedical Engineering.

[17]  Kazuyuki Aihara,et al.  Circuit Implementation and Dynamics of a Two-Dimensional MOSFET Neuron Model , 2007, Int. J. Bifurc. Chaos.

[18]  Christofer Toumazou,et al.  An analogue micropower log-domain silicon circuit for the Hodgkin and Huxley nerve axon , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[19]  Kyusun Choi,et al.  Folded cascode CMOS mixer design and optimization in 70 nm technology , 2005, 48th Midwest Symposium on Circuits and Systems, 2005..

[20]  P. Hasler,et al.  A CMOS floating-gate matrix transform imager , 2005, IEEE Sensors Journal.

[21]  J.N. Babanezhad,et al.  A 20-V four-quadrant CMOS analog multiplier , 1985, IEEE Journal of Solid-State Circuits.

[22]  P. Hasler,et al.  Adaptive Algorithm Using Hot-Electron Injection for Programming Analog Computational Memory Elements Within 0.2% of Accuracy Over 3.5 Decades , 2006, IEEE Journal of Solid-State Circuits.

[23]  Y. Tsividis Operation and modeling of the MOS transistor , 1987 .

[24]  Jörg Kramer,et al.  An integrated optical transient sensor , 2002 .

[25]  Bradley A. Minch,et al.  Synthesis of static and dynamic multiple-input translinear element networks , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[26]  Arindam Basu,et al.  A Charge-Based Low-Power High-SNR Capacitive Sensing Interface Circuit , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Johan H. Huijsing,et al.  Compact low-voltage power-efficient operational amplifier cells for VLSI , 1998, IEEE J. Solid State Circuits.

[28]  Gert Cauwenberghs,et al.  Floating-gate adaptation for focal-plane online nonuniformity correction , 2001 .

[29]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[30]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[31]  Paul E. Hasler,et al.  A bio-physically inspired silicon neuron , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[32]  Stephen P. DeWeerth,et al.  Analog VLSI circuits for stimulus localization and centroid computation , 1992, International Journal of Computer Vision.

[33]  L.A.D. van den Broeke,et al.  Wide-band integrated optical receiver with improved dynamic range using a current switch at the input , 1993 .

[34]  Kazuyuki Aihara,et al.  A MOSFET-based model of a class 2 nerve membrane , 2005, IEEE Transactions on Neural Networks.

[35]  Christopher M. Twigg,et al.  Programmable Conductance Switches for FPAAs , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[36]  Luigi Fortuna,et al.  A CNN-based chip for robot locomotion control , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[37]  Robert G. Meyer,et al.  A wideband low-noise variable-gain BiCMOS transimpedance amplifier , 1994 .

[38]  Arindam Basu,et al.  RASP 2.8: A new generation of floating-gate based field programmable analog array , 2008, 2008 IEEE Custom Integrated Circuits Conference.

[39]  Marek Wojcikowski,et al.  A field programmable analog array for CMOS continuous-time OTA-C filter applications , 2002 .

[40]  E. Sanchez-Sinencio,et al.  A current-mode based field programmable analog array architecture for signal processing applications , 1998, Proceedings of the IEEE 1998 Custom Integrated Circuits Conference (Cat. No.98CH36143).

[41]  Yannis Tsividis,et al.  A reconfigurable VLSI neural network , 1992 .

[42]  Yngvar Berg,et al.  Ultra low-voltage/low-power digital floating-gate circuits , 1999 .

[43]  Eve Marder,et al.  Mechanisms for oscillation and frequency control in reciprocally inhibitory model neural networks , 1994, Journal of Computational Neuroscience.

[44]  E.K.F. Lee,et al.  Field programmable analogue array based on mosfet transconductors , 1992 .

[45]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[46]  Ralph Etienne-Cummings,et al.  A switched capacitor implementation of the generalized linear integrate-and-fire neuron , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[47]  Ralph Etienne-Cummings,et al.  A Silicon Central Pattern Generator Controls Locomotion in Vivo , 2008, IEEE Transactions on Biomedical Circuits and Systems.

[48]  G. Scandurra,et al.  Improved trade-off between noise and bandwidth in op-amp based transimpedance amplifier , 2004, Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.04CH37510).

[49]  B.K. Ahuja,et al.  A very high precision 500-nA CMOS floating-gate analog voltage reference , 2005, IEEE Journal of Solid-State Circuits.

[50]  Venkatesh Srinivasan Programmable Analog Techniques For Precision Analog Circuits, Low-Power Signal Processing and On-Chip Learning , 2006 .

[51]  S. Espejo,et al.  A 1000 FPS at 128/spl times/128 vision processor with 8-bit digitized I/O , 2004, IEEE Journal of Solid-State Circuits.

[52]  Arindam Basu,et al.  Bifurcations in a silicon neuron , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[53]  Eugene M. Izhikevich,et al.  Weakly pulse-coupled oscillators, FM interactions, synchronization, and oscillatory associative memory , 1999, IEEE Trans. Neural Networks.

[54]  Stephen P. DeWeerth,et al.  A multiconductance silicon neuron with biologically matched dynamics , 2004, IEEE Transactions on Biomedical Engineering.

[55]  Paul Hasler,et al.  A family of floating-gate adapting synapses based upon transistor channel models , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[56]  G. Ermentrout,et al.  Analysis of neural excitability and oscillations , 1989 .

[57]  John D. Hyde,et al.  A 300-MS/s 14-bit digital-to-analog converter in logic CMOS , 2003, IEEE J. Solid State Circuits.

[58]  Tsung-Shuen Hung,et al.  A CMOS infrared wireless optical receiver front-end with a variable-gain fully-differential transimpedance amplifier , 2005, IEEE Transactions on Consumer Electronics.

[59]  Arindam Basu,et al.  A Floating-gate Based Low-Power Capacitive Sensing Interface Circuit , 2006, IEEE Custom Integrated Circuits Conference 2006.

[60]  V. Douence,et al.  Design of an analogue ASIC using subthreshold CMOS transistors to model biological neurons , 2001, Proceedings of the IEEE 2001 Custom Integrated Circuits Conference (Cat. No.01CH37169).

[61]  Kari Halonen,et al.  A mixed-mode polynomial cellular array processor hardware realization , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[62]  A.J. Preyer,et al.  Causes of Transient Instabilities in the Dynamic Clamp , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[63]  Takashi Morie,et al.  A PWM analog memory programming circuit for floating-gate MOSFETs with 75-μs programming time and 11-bit updating resolution , 2001, IEEE J. Solid State Circuits.

[64]  Paul E. Hasler,et al.  A Precision CMOS Amplifier Using Floating-Gate Transistors for Offset Cancellation , 2007, IEEE Journal of Solid-State Circuits.

[65]  J. M. Rochelle,et al.  A wide-range logarithmic electrometer with improved accuracy and temperature stability , 1992 .

[66]  Sheng-Yu Peng,et al.  PS-15 Floating-Gate Based CMUT Sensing Circuit Using Capacitive Feedback Charge Amplifier , 2006, 2006 IEEE Ultrasonics Symposium.

[67]  Paul E. Hasler,et al.  Indirect Programming of Floating-Gate Transistors , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[68]  David V. Anderson,et al.  Application of a human auditory model to loudness perception and hearing compensation , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[69]  E. J. Doedel,et al.  AUTO: a program for the automatic bifurcation analysis of autonomous systems , 1980 .

[70]  E.J. Brauer,et al.  Real-time interaction between a neuromorphic electronic circuit and the spinal cord , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[71]  Paul E. Hasler,et al.  A Low-Power Programmable Bandpass Filter Section for Higher Order Filter Applications , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[72]  Christopher M. Twigg,et al.  A Large-Scale Reconfigurable Analog Signal Processor (RASP) IC , 2006, IEEE Custom Integrated Circuits Conference 2006.

[73]  C. Mead,et al.  White noise in MOS transistors and resistors , 1993, IEEE Circuits and Devices Magazine.

[74]  Arindam Basu,et al.  A Low-Power, Compact, Adaptive Logarithmic Transimpedance Amplifier Operating Over Seven Decades of Current , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[75]  Arindam Basu,et al.  Transistor channel dendrites implementing HMM classifiers , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[76]  David A. Johns,et al.  A CMOS optical preamplifier for wireless infrared communications , 1999 .

[77]  Kazuyuki Aihara,et al.  MOSFET implementation of class I* neurons coupled by gap junctions , 2005, Artificial Life and Robotics.

[78]  Rahul Sarpeshkar,et al.  Efficient precise computation with noisy components: extrapolating from an electronic cochlea to the brain , 1997 .

[79]  Luigi Fortuna,et al.  A CNN-based chip for robot locomotion control , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[80]  Gustavo Liñán Cembrano,et al.  A 1000 FPS at 128×128 vision processor with 8-bit digitized I/O , 2004, IEEE J. Solid State Circuits.

[81]  Paul E. Hasler,et al.  Offset compensation in flash ADCs using floating-gate circuits , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[82]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[83]  Gert Cauwenberghs,et al.  Analog VLSI neuromorphic network with programmable membrane channel kinetics , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[84]  Stephen P. DeWeerth,et al.  Adaptation in a VLSI model of a neuron , 1999 .

[85]  Csaba Petre,et al.  Automated conversion of Simulink designs to analog hardware on an FPAA , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[86]  Arindam Basu,et al.  A large-scale Reconfigurable Smart Sensory Chip , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[87]  H. Van Tran,et al.  A 2.5 V 256-level non-volatile analog storage device using EEPROM technology , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[88]  Kwabena Boahen,et al.  Synchrony in Silicon: The Gamma Rhythm , 2007, IEEE Transactions on Neural Networks.

[89]  R. Douglas,et al.  A silicon neuron , 1991, Nature.

[90]  Gert Cauwenberghs,et al.  Fixed-current method for programming large floating-gate arrays , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[91]  Pamela Abshire,et al.  A floating-gate comparator with automatic offset adaptation for 10-bit data conversion , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[92]  Paul E. Hasler,et al.  Programmable $G_{m}$– $C$ Filters Using Floating-Gate Operational Transconductance Amplifiers , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[93]  Tobi Delbrück,et al.  Adaptive photoreceptor with wide dynamic range , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[94]  G. Serrano,et al.  A Precision Low-TC Wide-Range CMOS Current Reference , 2008, IEEE Journal of Solid-State Circuits.

[95]  Yannis P. Tsividis,et al.  A Reconfigurable Analog VLSI Neural Network Chip , 1989, NIPS.

[96]  Eugene M. Izhikevich,et al.  Neural excitability, Spiking and bursting , 2000, Int. J. Bifurc. Chaos.

[97]  Michael K. Giles,et al.  A high speed centroid computation circuit in analog VLSI , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[98]  S. Renaud,et al.  A Conductance-Based Silicon Neuron with Dynamically Tunable Model Parameters , 2005, Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering, 2005..

[99]  D.H.K. Hoe,et al.  An auto-ranging photodiode preamplifier with 114 dB dynamic range , 1996 .

[100]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[101]  Grégory Faye,et al.  An introduction to bifurcation theory , 2011 .

[102]  C. Lyden,et al.  Op-amp based CMOS field-programmable analogue array , 2000 .

[103]  M. Ortmanns,et al.  A Field-Programmable Analog Array of 55 Digitally Tunable OTAs in a Hexagonal Lattice , 2008, IEEE Journal of Solid-State Circuits.

[104]  Stephen P. DeWeerth,et al.  Bifurcation Analysis of a Silicon Neuron , 1999, NIPS.

[105]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[106]  Christopher M. Twigg,et al.  A Floating-Gate-Based Programmable CMOS Reference , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[107]  David V. Anderson,et al.  Placement for large-scale floating-gate field-programable analog arrays , 2006, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[108]  Shui-Nee Chow,et al.  An analysis of phase noise and Fokker-Planck equations , 2007 .

[109]  Tobi Delbrück,et al.  Self-biasing low power adaptive photoreceptor , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[110]  Arindam Basu,et al.  Dynamics of a Logarithmic Transimpedance Amplifier , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[111]  A. Hodgkin The local electric changes associated with repetitive action in a non‐medullated axon , 1948, The Journal of physiology.

[112]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .