Biological plausibility and stochasticity in scalable VO2 active memristor neurons

Neuromorphic networks of artificial neurons and synapses can solve computationally hard problems with energy efficiencies unattainable for von Neumann architectures. For image processing, silicon neuromorphic processors outperform graphic processing units in energy efficiency by a large margin, but deliver much lower chip-scale throughput. The performance-efficiency dilemma for silicon processors may not be overcome by Moore’s law scaling of silicon transistors. Scalable and biomimetic active memristor neurons and passive memristor synapses form a self-sufficient basis for a transistorless neural network. However, previous demonstrations of memristor neurons only showed simple integrate-and-fire behaviors and did not reveal the rich dynamics and computational complexity of biological neurons. Here we report that neurons built with nanoscale vanadium dioxide active memristors possess all three classes of excitability and most of the known biological neuronal dynamics, and are intrinsically stochastic. With the favorable size and power scaling, there is a path toward an all-memristor neuromorphic cortical computer.The neuromorphic computing based on complementary metal-oxide-semiconductor transistors holds promise for artificial intelligence, but it suffers from the trade-off between scalability and biological fidelity. Yi et al. emulate 23 types of biological neuronal behaviors using scalable VO2 active memristors.

[1]  Karin Hinzer,et al.  Encoding with Bursting, Subthreshold Oscillations, and Noise in Mammalian Cold Receptors , 1996, Neural Computation.

[2]  C. N. Berglund,et al.  Electronic Properties of V O 2 near the Semiconductor-Metal Transition , 1969 .

[3]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[4]  R Stanley Williams,et al.  Sub-100 fJ and sub-nanosecond thermally driven threshold switching in niobium oxide crosspoint nanodevices , 2012, Nanotechnology.

[5]  Jean-Christophe Orlianges,et al.  Current-induced electrical self-oscillations across out-of-plane threshold switches based on VO2 layers integrated in crossbars geometry , 2014 .

[6]  Andrew S. Cassidy,et al.  Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  Leon O. Chua,et al.  Neurons are Poised Near the Edge of Chaos , 2012, Int. J. Bifurc. Chaos.

[8]  Eugene M. Izhikevich,et al.  Neural excitability, Spiking and bursting , 2000, Int. J. Bifurc. Chaos.

[9]  Chung Lam,et al.  Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array , 2014, Front. Neurosci..

[10]  S. Ramanathan,et al.  Oxide Electronics Utilizing Ultrafast Metal-Insulator Transitions , 2011 .

[11]  Peng Lin,et al.  Fully memristive neural networks for pattern classification with unsupervised learning , 2018 .

[12]  Patrick Georges,et al.  Femtosecond laser excitation of the semiconductor‐metal phase transition in VO2 , 1994 .

[13]  Biswa Sengupta,et al.  The effect of cell size and channel density on neuronal information encoding and energy efficiency , 2013, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[14]  Stephen J. Hudgens,et al.  Invited paper: Thin-film Ovonic threshold switch: Its operation and application in modern integrated circuits , 2012, Electronic Materials Letters.

[15]  M. Rozenberg,et al.  A Leaky‐Integrate‐and‐Fire Neuron Analog Realized with a Mott Insulator , 2017 .

[16]  Hewitt D. Crane.  The Neuristor , 1960, IRE Trans. Electron. Comput..

[17]  Volker Eyert,et al.  The metal‐insulator transitions of VO2: A band theoretical approach , 2002, Annalen der Physik.

[18]  Shriram Ramanathan,et al.  Thermal conductivity and dynamic heat capacity across the metal-insulator transition in thin film VO2 , 2010 .

[19]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[20]  R BoerdeF.,et al.  Electronic properties of U2Cu9Al , 1999 .

[21]  A. Cavalleri,et al.  Femtosecond Structural Dynamics in VO2 during an Ultrafast Solid-Solid Phase Transition. , 2001, Physical review letters.

[22]  Giacomo Indiveri,et al.  A current-mode conductance-based silicon neuron for address-event neuromorphic systems , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[23]  Warren Robinett,et al.  Memristor-CMOS hybrid integrated circuits for reconfigurable logic. , 2009, Nano letters.

[24]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[25]  M. Bialko,et al.  Novel inductorless neuristor line , 1975 .

[26]  Nabil H. Farhat,et al.  Bifurcating neuron: characterization and dynamics , 1993, Optics + Photonics.

[27]  Doo Seok Jeong,et al.  Reliability of neuronal information conveyed by unreliable neuristor-based leaky integrate-and-fire neurons: a model study , 2015, Scientific Reports.

[28]  황현상.  High density neuromorphic system with Mo/Pr0.7Ca0.3MnO3 synapse and NbO2 IMT oscillator neuron , 2015 .

[29]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[30]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[31]  Narayan Srinivasa,et al.  Energy-Efficient Neuron, Synapse and STDP Integrated Circuits , 2012, IEEE Transactions on Biomedical Circuits and Systems.

[32]  Savel ' Ev.  Current-Controlled Negative Differential Resistance Due to Joule Heating In Tio2 , 2012 .

[33]  J. NAGUMOt,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 2006 .

[34]  Wei D. Lu,et al.  Experimental Demonstration of Feature Extraction and Dimensionality Reduction Using Memristor Networks. , 2017, Nano letters.

[35]  S. Ovshinsky.  Reversible Electrical Switching Phenomena in Disordered Structures , 1968 .

[36]  Piotr Dudek,et al.  Compact silicon neuron circuit with spiking and bursting behaviour , 2008, Neural Networks.

[37]  T. Ando,et al.  CMOS compatible MIM decoupling capacitor with reliable sub-nm EOT high-k stacks for the 7 nm node and beyond , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[38]  T. Prodromakis,et al.  High Density Crossbar Arrays with Sub- 15 nm Single Cells via Liftoff Process Only , 2016, Scientific Reports.

[39]  Leon O. Chua,et al.  Memristor Emulators: A Note on Modeling , 2017 .

[40]  Leon O. Chua,et al.  Hodgkin-Huxley Axon is Made of memristors , 2012, Int. J. Bifurc. Chaos.

[41]  R. M. Zilberstein.  "Neuristor—A novel device and system concept" , 1963 .

[42]  Daniele Ielmini,et al.  Threshold switching mechanism by high-field energy gain in the hopping transport of chalcogenide glasses , 2008 .

[43]  Jongkil Park,et al.  A 65k-neuron 73-Mevents/s 22-pJ/event asynchronous micro-pipelined integrate-and-fire array transceiver , 2014, 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings.

[44]  Karin Strauss,et al.  Accelerating Deep Convolutional Neural Networks Using Specialized Hardware , 2015 .

[45]  A. Hodgkin.  The local electric changes associated with repetitive action in a non‐medullated axon , 1948, The Journal of physiology.

[46]  Frank C. Hoppensteadt,et al.  Bursts as a unit of neural information: selective communication via resonance , 2003, Trends in Neurosciences.

[47]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[48]  Michael Stiber,et al.  PERIODIC INHIBITION OF LIVING PACEMAKER NEURONS (I): LOCKED, INTERMITTENT, MESSY, AND HOPPING BEHAVIORS , 1991 .

[49]  S. Joshi,et al.  65k-neuron integrate-and-fire array transceiver with address-event reconfigurable synaptic routing , 2012, 2012 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[50]  M. Pickett,et al.  A scalable neuristor built with Mott memristors. , 2013, Nature materials.

[51]  J. Rothman,et al.  The roles potassium currents play in regulating the electrical activity of ventral cochlear nucleus neurons. , 2003, Journal of neurophysiology.

[52]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[53]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[54]  David C. Tam,et al.  A joint interspike interval difference stochastic spike train analysis: detecting local trends in the temporal firing patterns of single neurons , 1999, Biological Cybernetics.

[55]  J. Yang,et al.  Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. , 2017, Nature materials.

[56]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[57]  John Rinzel,et al.  TYPE III EXCITABILITY, SLOPE SENSITIVITY AND COINCIDENCE DETECTION. , 2012, Discrete and continuous dynamical systems. Series A.

[58]  Manuel Le Gallo,et al.  Stochastic phase-change neurons. , 2016, Nature nanotechnology.

[59]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[60]  Sungho Kim,et al.  Experimental demonstration of a second-order memristor and its ability to biorealistically implement synaptic plasticity. , 2015, Nano letters.

[61]  Arnaud Magrez,et al.  Elevated transition temperature in Ge doped VO2 thin films , 2017 .

[62]  GPU-Based Deep Learning Inference: A Performance and Power Analysis , 2015 .

[63]  Leon O. Chua,et al.  Circuit Elements With Memory: Memristors, Memcapacitors, and Meminductors , 2009, Proceedings of the IEEE.

[64]  Kae Nakamura,et al.  Predictive Reward Signal of Dopamine Neurons , 2015 .

[65]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[66]  Olivier Temam,et al.  Hardware spiking neurons design: Analog or digital? , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[67]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[68]  G. Ermentrout,et al.  Analysis of neural excitability and oscillations , 1989 .

[69]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..