PrxCa1−xMnO3 based stochastic neuron for Boltzmann machine to solve “maximum cut” problem

The neural network enables efficient solutions for Nondeterministic Polynomial-time (NP) hard problems, which are challenging for conventional von Neumann computing. The hardware implementation, i.e., neuromorphic computing, aspires to enhance this efficiency by custom hardware. Particularly, NP hard graphical constraint optimization problems are solved by a network of stochastic binary neurons to form a Boltzmann Machine (BM). The implementation of stochastic neurons in hardware is a major challenge. In this work, we demonstrate that the high to low resistance switching (set) process of a PrxCa1−xMnO3 (PCMO) based RRAM (Resistive Random Access Memory) is probabilistic. Additionally, the voltage-dependent probability distribution approximates a sigmoid function with 1.35%–3.5% error. Such a sigmoid function is required for a BM. Thus, the Analog Approximate Sigmoid (AAS) stochastic neuron is proposed to solve the maximum cut—an NP hard problem. It is compared with Digital Precision-controlled Sigmoid (DPS) implementation using (a) pure CMOS design and (b) hybrid (RRAM integrated with CMOS). The AAS design solves the problem with 98% accuracy, which is comparable with the DPS design but with 10× area and 4× energy advantage. Thus, ASIC neuro-processors based on novel analog neuromorphic devices based BM are promising for efficiently solving large scale NP hard optimization problems.The neural network enables efficient solutions for Nondeterministic Polynomial-time (NP) hard problems, which are challenging for conventional von Neumann computing. The hardware implementation, i.e., neuromorphic computing, aspires to enhance this efficiency by custom hardware. Particularly, NP hard graphical constraint optimization problems are solved by a network of stochastic binary neurons to form a Boltzmann Machine (BM). The implementation of stochastic neurons in hardware is a major challenge. In this work, we demonstrate that the high to low resistance switching (set) process of a PrxCa1−xMnO3 (PCMO) based RRAM (Resistive Random Access Memory) is probabilistic. Additionally, the voltage-dependent probability distribution approximates a sigmoid function with 1.35%–3.5% error. Such a sigmoid function is required for a BM. Thus, the Analog Approximate Sigmoid (AAS) stochastic neuron is proposed to solve the maximum cut—an NP hard problem. It is compared with Digital Precision-controlled Sigmoid (DPS...

[1]  D. Bounds.  New optimization methods from physics and biology , 1987, Nature.

[2]  Emile H. L. Aarts,et al.  Combinatorial Optimization on a Boltzmann Machine , 1989, J. Parallel Distributed Comput..

[3]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[4]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[5]  Michael J. Berry,et al.  Refractoriness and Neural Precision , 1997, The Journal of Neuroscience.

[6]  Rahul Sarpeshkar,et al.  Analog Versus Digital: Extrapolating from Electronics to Neurobiology , 1998, Neural Computation.

[7]  G. Mur,et al.  Least-squares minimising finite-element formulation for static and stationary electric and magnetic fields , 1998 .

[8]  Kate A. Smith,et al.  Neural Networks for Combinatorial Optimization: a Review of More Than a Decade of Research , 1999 .

[9]  A. Joshi,et al.  The traveling salesman problem: A hierarchical model , 2000, Memory & cognition.

[10]  H. Seung,et al.  Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[11]  W. Lu,et al.  Programmable Resistance Switching in Nanoscale Two-terminal Devices , 2008 .

[12]  Derek Abbott,et al.  What Is Stochastic Resonance? Definitions, Misconceptions, Debates, and Its Relevance to Biology , 2009, PLoS Comput. Biol..

[13]  Rui-Sheng Wang,et al.  Maximum cut in fuzzy nature: Models and algorithms , 2010, J. Comput. Appl. Math..

[14]  H. Hwang,et al.  Low programming voltage resistive switching in reactive metal/polycrystalline Pr0.7Ca0.3MnO3 devices , 2010 .

[15]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[16]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[17]  A. Thomas,et al.  The Memristive Magnetic Tunnel Junction as a Nanoscopic Synapse‐Neuron System , 2012, Advanced materials.

[18]  Stefan Habenschuss,et al.  Stochastic Computations in Cortical Microcircuit Models , 2013, PLoS Comput. Biol..

[19]  Jiantao Zhou,et al.  Stochastic Memristive Devices for Computing and Neuromorphic Applications , 2013, Nanoscale.

[20]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[21]  T. Prodromakis,et al.  Stochastic switching of TiO2-based memristive devices with identical initial memory states , 2014, Nanoscale Research Letters.

[22]  Dhireesha Kudithipudi,et al.  A current-mode CMOS/memristor hybrid implementation of an extreme learning machine , 2014, GLSVLSI '14.

[23]  Gert Cauwenberghs,et al.  Memristors Empower Spiking Neurons With Stochasticity , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[24]  N. Panwar,et al.  Effect of Morphological Change on Unipolar and Bipolar Switching Characteristics in Pr 0.7 Ca 0.3 MnO 3 Based RRAM , 2015 .

[25]  E. Vianello,et al.  HfO2-Based OxRAM Devices as Synapses for Convolutional Neural Networks , 2015, IEEE Transactions on Electron Devices.

[26]  Jongin Kim,et al.  Electronic system with memristive synapses for pattern recognition , 2015, Scientific Reports.

[27]  Giacomo Indiveri,et al.  Memory and Information Processing in Neuromorphic Systems , 2015, Proceedings of the IEEE.

[28]  Manuel Le Gallo,et al.  Stochastic phase-change neurons. , 2016, Nature nanotechnology.

[29]  Kaushik Roy,et al.  Magnetic Tunnel Junction Mimics Stochastic Cortical Spiking Neurons , 2015, Scientific Reports.

[30]  Robert A. Nawrocki,et al.  A Mini Review of Neuromorphic Architectures and Implementations , 2016, IEEE Transactions on Electron Devices.

[31]  Arash Ahmadi,et al.  Realistic Hodgkin–Huxley Axons Using Stochastic Behavior of Memristors , 2017, Neural Processing Letters.

[32]  Yu Wang,et al.  Leveraging Stochastic Memristor Devices in Neuromorphic Hardware Systems , 2016, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[33]  Stefan Habenschuss,et al.  Solving Constraint Satisfaction Problems with Networks of Spiking Neurons , 2016, Front. Neurosci..

[34]  Rawan Naous,et al.  Stochasticity Modeling in Memristors , 2016, IEEE Transactions on Nanotechnology.

[35]  John Paul Strachan,et al.  Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing , 2017, Nature.

[36]  N. Mohapatra,et al.  Leaky Integrate and Fire Neuron by Charge-Discharge Dynamics in Floating-Body MOSFET , 2017, Scientific Reports.

[37]  N. Panwar,et al.  PCMO-Based RRAM and NPN Bipolar Selector as Synapse for Energy Efficient STDP , 2017, IEEE Electron Device Letters.

[38]  B. Rajendran,et al.  Arbitrary Spike Time Dependent Plasticity (STDP) in Memristor by Analog Waveform Engineering , 2017, IEEE Electron Device Letters.

[39]  Suman Datta,et al.  Vertex coloring of graphs via phase dynamics of coupled oscillatory networks , 2016, Scientific Reports.

[40]  Steve B. Furber,et al.  Using Stochastic Spiking Neural Networks on SpiNNaker to Solve Constraint Satisfaction Problems , 2017, Front. Neurosci..

[41]  Bipin Rajendran,et al.  Stochastic learning in deep neural networks based on nanoscale PCMO device characteristics , 2018, Neurocomputing.

[42]  N. Mohapatra,et al.  Transient Variability in SOI-Based LIF Neuron and Impact on Unsupervised Learning , 2018, IEEE Transactions on Electron Devices.

[43]  U. Ganguly,et al.  Ionic Transport Barrier Tuning by Composition in Pr1–xCaxMnO3-Based Selector-Less RRAM and Its Effect on Memory Performance , 2018, IEEE Transactions on Electron Devices.

[44]  U. Ganguly,et al.  Transient Joule Heating-Based Oscillator Neuron for Neuromorphic Computing , 2018, IEEE Electron Device Letters.

[45]  Wei Yi,et al.  Biological plausibility and stochasticity in scalable VO2 active memristor neurons , 2018, Nature Communications.

[46]  Punyashloka Debashis,et al.  Design of Stochastic Nanomagnets for Probabilistic Spin Logic , 2018, IEEE Magnetics Letters.

[47]  U. Ganguly,et al.  PCMO RRAM for Integrate-and-Fire Neuron in Spiking Neural Networks , 2018, IEEE Electron Device Letters.

[48]  Kailash Chander,et al.  65nm Low Power Digital to Analog Converter for CUWB , 2018, 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI).

[49]  Supriyo Datta,et al.  Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons , 2018, ACM J. Emerg. Technol. Comput. Syst..

[50]  Baoshun Zhang,et al.  Voltage-Controlled Spintronic Stochastic Neuron Based on a Magnetic Tunnel Junction , 2019, Physical Review Applied.

[51]  N. Panwar,et al.  Temperature Effects in SET/RESET Voltage–Time Dilemma in Pr0.7Ca0.3MnO3-Based RRAM , 2019, IEEE Transactions on Electron Devices.

[52]  Supriyo Datta,et al.  Low-Barrier Magnet Design for Efficient Hardware Binary Stochastic Neurons , 2019, IEEE Magnetics Letters.