Switched-capacitor realization of presynaptic short-term-plasticity and stop-learning synapses in 28 nm CMOS

Synaptic dynamics, such as long- and short-term plasticity, play an important role in the complexity and biological realism achievable when running neural networks on a neuromorphic IC. For example, they endow the IC with an ability to adapt and learn from its environment. In order to achieve the millisecond to second time constants required for these synaptic dynamics, analog subthreshold circuits are usually employed. However, due to process variation and leakage problems, it is almost impossible to port these types of circuits to modern sub-100nm technologies. In contrast, we present a neuromorphic system in a 28 nm CMOS process that employs switched capacitor (SC) circuits to implement 128 short term plasticity presynapses as well as 8192 stop-learning synapses. The neuromorphic system consumes an area of 0.36 mm2 and runs at a power consumption of 1.9 mW. The circuit makes use of a technique for minimizing leakage effects allowing for real-time operation with time constants up to several seconds. Since we rely on SC techniques for all calculations, the system is composed of only generic mixed-signal building blocks. These generic building blocks make the system easy to port between technologies and the large digital circuit part inherent in an SC system benefits fully from technology scaling.

[1]  Stephan Henker,et al.  A 32 GBit/s communication SoC for a waferscale neuromorphic system , 2012, Integr..

[2]  Giacomo Indiveri,et al.  A VLSI spike-driven dynamic synapse which learns only when necessary , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[3]  René Schüffny,et al.  Transient responses of activity-dependent synapses to modulated pulse trains , 2009, Neurocomputing.

[4]  Davide Badoni,et al.  Spike-Driven Synaptic Plasticity: Theory, Simulation, VLSI Implementation , 2000, Neural Computation.

[5]  Chiara Bartolozzi,et al.  Ultra low leakage synaptic scaling circuits for implementing homeostatic plasticity in neuromorphic architectures , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[6]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[7]  René Schüffny,et al.  Replicating experimental spike and rate based neural learning in CMOS , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[8]  Fabien Alibart,et al.  A Memristive Nanoparticle/Organic Hybrid Synapstor for Neuroinspired Computing , 2011, ArXiv.

[9]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[10]  Olaf Sporns,et al.  Neurobiologically Realistic Determinants of Self-Organized Criticality in Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[11]  Giacomo Indiveri,et al.  An Event-Based Neural Network Architecture With an Asynchronous Programmable Synaptic Memory , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[12]  P.R. Kinget Device mismatch and tradeoffs in the design of analog circuits , 2005, IEEE Journal of Solid-State Circuits.

[13]  Giacomo Indiveri,et al.  Spike-based learning with a generalized integrate and fire silicon neuron , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[14]  Giacomo Indiveri,et al.  Mapping arbitrary mathematical functions and dynamical systems to neuromorphic VLSI circuits for spike-based neural computation , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[15]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[16]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[17]  Andreas Kaiser,et al.  Very low-voltage fully differential amplifier for switched-capacitor applications , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[18]  Joaquín J. Torres,et al.  Maximum Memory Capacity on Neural Networks with Short-Term Synaptic Depression and Facilitation , 2009, Neural Computation.

[19]  Marko Noack,et al.  Configurable analog-digital conversion using the neural engineering framework , 2014, Front. Neurosci..

[20]  Alister Hamilton,et al.  Analog VLSI Circuit Implementation of an Adaptive Neuromorphic Olfaction Chip , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[22]  Stephan Henker,et al.  Highly integrated packet-based AER communication infrastructure with 3Gevent/S throughput , 2010, 2010 17th IEEE International Conference on Electronics, Circuits and Systems.

[23]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[24]  Tobi Delbrück,et al.  Addressable current reference array with 170dB dynamic range , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[25]  Gabor C. Temes,et al.  Circuit techniques for reducing the effects of op-amp imperfections: autozeroing, correlated double sampling, and chopper stabilization , 1996, Proc. IEEE.

[26]  H. Melchior,et al.  Four-quadrant CMOS analogue multiplier , 1984 .

[27]  Stephan Henker,et al.  Mapping Complex, Large – Scale Spiking Networks on Neural VLSI , 2007 .

[28]  Andrew S. Cassidy,et al.  A combinational digital logic approach to STDP , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[29]  Gustavo Deco,et al.  Network Bursting Dynamics in Excitatory Cortical Neuron Cultures Results from the Combination of Different Adaptive Mechanism , 2013, PloS one.

[30]  C. Mayr,et al.  Nonvolatile Multilevel Resistive Switching in $ \hbox{Ar}^{+}$ Irradiated $\hbox{BiFeO}_{3}$ Thin Films , 2013, IEEE Electron Device Letters.

[31]  Johannes Partzsch,et al.  Rate and Pulse Based Plasticity Governed by Local Synaptic State Variables , 2010, Front. Syn. Neurosci..

[32]  Ruey-Beei Wu,et al.  Differential Signaling , 2000 .

[33]  H. Kawaguchi,et al.  Managing subthreshold leakage in charge-based analog circuits with low-V/sub TH/ transistors by analog T- switch (AT-switch) and super cut-off CMOS (SCCMOS) , 2006, IEEE Journal of Solid-State Circuits.

[34]  Ralph Etienne-Cummings,et al.  A CMOS switched capacitor implementation of the Mihalas-Niebur neuron , 2009, 2009 IEEE Biomedical Circuits and Systems Conference.

[35]  René Schüffny,et al.  Synapse dynamics in CMOS derived from a model of neurotransmitter release , 2011, 2011 20th European Conference on Circuit Theory and Design (ECCTD).

[36]  C. Mayr,et al.  Active Pixel Sensor Arrays in 90 / 65 nm CMOS-Technologies with vertically stacked photodiodes , 2007 .

[37]  Mohammed Ismail,et al.  A nonlinear CMOS analog cell for VLSI signal and information processing , 1991 .

[38]  E. Rolls,et al.  Holding Multiple Items in Short Term Memory: A Neural Mechanism , 2013, PloS one.

[39]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[40]  Stephan Henker,et al.  Biology-derived synaptic dynamics and optimized system architecture for neuromorphic hardware , 2010, Proceedings of the 17th International Conference Mixed Design of Integrated Circuits and Systems - MIXDES 2010.

[41]  Stephan Henker,et al.  A novel ADPLL design using successive approximation frequency control , 2009, Microelectron. J..

[42]  Bang-Sup Song,et al.  A 10-b 20-Msample/s low-power CMOS ADC , 1995, IEEE J. Solid State Circuits.

[43]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[44]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Saibal Mukhopadhyay,et al.  Leakage current mechanisms and leakage reduction techniques in deep-submicrometer CMOS circuits , 2003, Proc. IEEE.

[46]  Marian Stamp Dawkins,et al.  The Noisy Brain: Stochastic Dynamics as a Principle of Brain Function The Noisy Brain: Stochastic Dynamics as a Principle of Brain Function. By Edmund T. Rolls & Gustavo Deco. Oxford: Oxford University Press (2010). Pp. 310. Price £37.95 hardback. , 2010, Animal Behaviour.

[47]  William J Spain,et al.  Synaptic depression as a timing device. , 2005, Physiology.

[48]  Chiara Bartolozzi,et al.  Synaptic Dynamics in Analog VLSI , 2007, Neural Computation.

[49]  Steve B. Furber,et al.  A hierachical configuration system for a massively parallel neural hardware platform , 2012, CF '12.

[50]  Jongkil Park,et al.  A 65k-neuron 73-Mevents/s 22-pJ/event asynchronous micro-pipelined integrate-and-fire array transceiver , 2014, 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings.

[51]  Nan Du,et al.  Waveform Driven Plasticity in BiFeO3 Memristive Devices: Model and Implementation , 2012, NIPS.

[52]  M. Schultz,et al.  A switched-capacitor implementation of short-term synaptic dynamics , 2012, Proceedings of the 19th International Conference Mixed Design of Integrated Circuits and Systems - MIXDES 2012.

[53]  René Schüffny,et al.  A Fast-Locking ADPLL With Instantaneous Restart Capability in 28-nm CMOS Technology , 2013, IEEE Transactions on Circuits and Systems II: Express Briefs.

[54]  Elijah Mwangi A FOUR QUADRANT CMOS ANALOGUE MULTIPLIER , 2016 .