Effects of memristive synapse radiation interactions on learning in spiking neural networks

This study uses advanced modeling and simulation to explore the effects of external events such as radiation interactions on the synaptic devices in an electronic spiking neural network. Specifically, the networks are trained using the spike-timing-dependent plasticity (STDP) learning rule to recognize spatio-temporal patterns (STPs) representing 25 and 100-pixel characters. Memristive synapses based on a TiO2 non-linear drift model designed in Verilog-A are utilized, with STDP learning behavior achieved through bi-phasic pre- and post-synaptic action potentials. The models are modified to include experimentally observed state-altering and ionizing radiation effects on the device. It is found that radiation interactions tend to make the connection between afferents stronger by increasing the conductance of synapses overall, subsequently distorting the STDP learning curve. In the absence of consistent STPs, these effects accumulate over time and make the synaptic weight evolutions unstable. With STPs at lower flux intensities, the network can recover and relearn with constant training. However, higher flux can overwhelm the leaky integrate-and-fire post-synaptic neuron circuits and reduce stability of the network.

[1]  Timothée Masquelier,et al.  STDP-based spiking deep neural networks for object recognition , 2016, Neural Networks.

[2]  M. Mitkova,et al.  Ionizing Radiation Effects on Nonvolatile Memory Properties of Programmable Metallization Cells , 2014, IEEE Transactions on Nuclear Science.

[3]  Michael N. Kozicki,et al.  Review of radiation effects on ReRAM devices and technology , 2017 .

[4]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[5]  Effects of 14 MeV neutron irradiation on the DC characteristics of CBRAM cells , 2016, 2016 16th European Conference on Radiation and Its Effects on Components and Systems (RADECS).

[6]  Michiel Steyaert,et al.  Radiation Effects in CMOS Technology , 2018 .

[7]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[8]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[10]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[11]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[12]  Andrew S. Keys,et al.  High‐Performance, Radiation‐Hardened Electronics for Space and Lunar Environments , 2008 .

[13]  Gopika Vinod,et al.  Application of artificial neural networks to nuclear power plant transient diagnosis , 2007, Reliab. Eng. Syst. Saf..

[14]  Robert C. Ivans,et al.  Modeling Memristor Radiation Interaction Events and the Effect on Neuromorphic Learning Circuits , 2018, Proceedings of the International Conference on Neuromorphic Systems.

[15]  Kurtis D. Cantley,et al.  Radiation Effect on Learning Behavior in Memristor-Based Neuromorphic Circuit , 2019, 2019 IEEE 62nd International Midwest Symposium on Circuits and Systems (MWSCAS).

[16]  Leon O. Chua,et al.  Memristor Bridge Synapse-Based Neural Network and Its Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[17]  D. R. Alexander,et al.  Transient ionizing radiation effects in devices and circuits , 2003 .

[18]  Frank L. Lewis,et al.  Control of a nonholonomic mobile robot using neural networks , 1998, IEEE Trans. Neural Networks.

[19]  Yan Wang,et al.  Proton-based total-dose irradiation effects on Cu/HfO2:Cu/Pt ReRAM devices , 2010, Nanotechnology.

[20]  Evangelos Eleftheriou,et al.  Learning spatio-temporal patterns in the presence of input noise using phase-change memristors , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[21]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[22]  Wei Lu,et al.  Memristive computing devices and applications , 2017, Journal of Electroceramics.

[23]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[24]  Uri C. Weiser,et al.  TEAM: ThrEshold Adaptive Memristor Model , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[25]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[26]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[27]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Martin Vingron,et al.  Comparative 3'UTR Analysis Allows Identification of Regulatory Clusters that Drive Eph/ephrin Expression in Cancer Cell Lines , 2008, PloS one.

[29]  Anand Subramaniam,et al.  Spatio-temporal pattern recognition in neural circuits with memory-transistor-driven memristive synapses , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[30]  Alessandro Paccagnella,et al.  Ionizing radiation induced leakage current on ultra-thin gate oxides , 1997 .

[31]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  T. Serrano-Gotarredona,et al.  STDP and STDP variations with memristors for spiking neuromorphic learning systems , 2013, Front. Neurosci..

[33]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[34]  Dalibor Biolek,et al.  Simplified SPICE model of TiO , 2015, 2015 International Conference on Memristive Systems (MEMRISYS).

[35]  Jintao Yu,et al.  Memristive devices for computing: Beyond CMOS and beyond von Neumann , 2017, 2017 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC).

[36]  Robinson E. Pino,et al.  Analysis of dynamic linear and non-linear memristor device models for emerging neuromorphic computing hardware design , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[37]  Stephen J. Wolf,et al.  The elusive memristor: properties of basic electrical circuits , 2008, 0807.3994.

[38]  Rachael D. Seidler,et al.  A simple solution for model comparison in bold imaging: the special case of reward prediction error and reward outcomes , 2013, Front. Neurosci..

[39]  E. E. King,et al.  Radiation Hardness of ${\rm TiO}_{2}$ Memristive Junctions , 2010, IEEE Transactions on Nuclear Science.

[40]  Darío Baptista,et al.  A survey of software and hardware use in artificial neural networks , 2013, Neural Computing and Applications.

[41]  H. Barnaby,et al.  Impact of Alpha Particles on the Electrical Characteristics of TiO$_{2}$ Memristors , 2011, IEEE Transactions on Nuclear Science.

[42]  N. Spruston,et al.  Frontiers in Synaptic Neuroscience Synaptic Neuroscience , 2022 .

[43]  Matthew D. Pickett,et al.  SPICE modeling of memristors , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[44]  B. Rajendran,et al.  Arbitrary Spike Time Dependent Plasticity (STDP) in Memristor by Analog Waveform Engineering , 2017, IEEE Electron Device Letters.

[45]  L. Xing,et al.  B cells inhibit bone formation in rheumatoid arthritis by suppressing osteoblast differentiation , 2018, Nature Communications.

[46]  G. Subramanyam,et al.  A Memristor Device Model , 2011, IEEE Electron Device Letters.

[47]  R. Lacoe Improving Integrated Circuit Performance Through the Application of Hardness-by-Design Methodology , 2008, IEEE Transactions on Nuclear Science.

[48]  Bernabé Linares-Barranco,et al.  On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex , 2011, Front. Neurosci..

[49]  B. Narasimham,et al.  Characterization of Digital Single Event Transient Pulse-Widths in 130-nm and 90-nm CMOS Technologies , 2007, IEEE Transactions on Nuclear Science.

[50]  Displacement Damage in TiO $_{2}$ Memristor Devices , 2013, IEEE Transactions on Nuclear Science.

[51]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[52]  Patrick R. Mickel,et al.  The Susceptibility of ${\hbox {TaO}}_{\rm x}$-Based Memristors to High Dose Rate Ionizing Radiation and Total Ionizing Dose , 2014, IEEE Transactions on Nuclear Science.

[53]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[54]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[55]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[56]  William M. Tong,et al.  Radiation effects studies on thin film TiO2 memristor devices , 2013, 2013 IEEE Aerospace Conference.

[57]  Kurtis D. Cantley,et al.  Learning Behavior of Memristor-Based Neuromorphic Circuits in the Presence of Radiation , 2019, ICONS.

[58]  P E Dodd,et al.  Current and Future Challenges in Radiation Effects on CMOS Electronics , 2010, IEEE Transactions on Nuclear Science.

[59]  Chris Yakopcic,et al.  Generalized Memristive Device SPICE Model and its Application in Circuit Design , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[60]  J.A. Felix,et al.  Radiation Effects in MOS Oxides , 2008, IEEE Transactions on Nuclear Science.

[61]  S. M. Dalton,et al.  Initial Assessment of the Effects of Radiation on the Electrical Characteristics of ${\rm TaO}_{\rm x}$ Memristive Memories , 2012, IEEE Transactions on Nuclear Science.

[62]  Mo Jamshidi,et al.  Stock market prediction by using artificial neural network , 2014, 2014 World Automation Congress (WAC).

[63]  Yusuf Leblebici,et al.  Neuromorphic system with phase-change synapses for pattern learning and feature extraction , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[64]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.