Memristive device based learning for navigation in robots

Biomimetic robots have gained attention recently for various applications ranging from resource hunting to search and rescue operations during disasters. Biological species are known to intuitively learn from the environment, gather and process data, and make appropriate decisions. Such sophisticated computing capabilities in robots are difficult to achieve, especially if done in real-time with ultra-low energy consumption. Here, we present a novel memristive device based learning architecture for robots. Two terminal memristive devices with resistive switching of oxide layer are modeled in a crossbar array to develop a neuromorphic platform that can impart active real-time learning capabilities in a robot. This approach is validated by navigating a robot vehicle in an unknown environment with randomly placed obstacles. Further, the proposed scheme is compared with reinforcement learning based algorithms using local and global knowledge of the environment. The simulation as well as experimental results corroborate the validity and potential of the proposed learning scheme for robots. The results also show that our learning scheme approaches an optimal solution for some environment layouts in robot navigation.

[1]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[2]  R. Shapley,et al.  Contrast's effect on spatial summation by macaque V1 neurons , 1999, Nature Neuroscience.

[3]  J. Kim,et al.  Current transport in metal/hafnium oxide/silicon structure , 2002, IEEE Electron Device Letters.

[4]  Thomas Schultz,et al.  Ultra-low energy neuromorphic device based navigation approach for biomimetic robots , 2016, 2016 IEEE National Aerospace and Electronics Conference (NAECON) and Ohio Innovation Summit (OIS).

[5]  Kaushik Roy,et al.  Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning , 2016, Scientific Reports.

[6]  Eugene M. Izhikevich,et al.  Resonate-and-fire neurons , 2001, Neural Networks.

[7]  Byoung Hun Lee,et al.  Nanoscale RRAM-based synaptic electronics: toward a neuromorphic computing device , 2013, Nanotechnology.

[8]  Jinfeng Kang,et al.  Metal oxide resistive random access memory based synaptic devices for brain-inspired computing , 2016 .

[9]  Yu Wang,et al.  Energy Efficient RRAM Spiking Neural Network for Real Time Classification , 2015, ACM Great Lakes Symposium on VLSI.

[10]  Bipin Rajendran,et al.  Novel synaptic memory device for neuromorphic computing , 2014, Scientific Reports.

[11]  Fei Zhou,et al.  Demonstration of Synaptic Behaviors and Resistive Switching Characterizations by Proton Exchange Reactions in Silicon Oxide , 2016, Scientific Reports.

[12]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[13]  H. Hwang,et al.  Optimized Programming Scheme Enabling Linear Potentiation in Filamentary HfO2 RRAM Synapse for Neuromorphic Systems , 2016, IEEE Transactions on Electron Devices.

[14]  Min-Chul Sun,et al.  Simulation Study on Silicon-Based Floating Body Synaptic Transistor with Short- and Long-Term Memory Functions and Its Spike Timing-Dependent Plasticity , 2016 .

[15]  Yukihiro Kaneko,et al.  Supervised Learning Using Spike-Timing-Dependent Plasticity of Memristive Synapses , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Branden Long,et al.  Doped HfO2 based nanoelectronic memristive devices for self-learning neural circuits and architecture , 2013, 2013 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[17]  Paul E. Hasler,et al.  Floating Gate Synapses With Spike-Time-Dependent Plasticity , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[18]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[19]  Chiara Bartolozzi,et al.  Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems , 2014, Proceedings of the IEEE.

[20]  Gianluca Antonelli,et al.  A calibration method for odometry of mobile robots based on the least-squares technique: theory and experimental validation , 2005, IEEE Transactions on Robotics.

[21]  E. Eleftheriou,et al.  All-memristive neuromorphic computing with level-tuned neurons , 2016, Nanotechnology.

[22]  Robert B. Fisher,et al.  Special issue on animal and insect behaviour understanding in image sequences , 2015, EURASIP J. Image Video Process..

[23]  John W. Backus,et al.  Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs , 1978, CACM.

[24]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[25]  E. Schuman,et al.  Long-lasting neurotrophin-induced enhancement of synaptic transmission in the adult hippocampus , 1995, Science.

[26]  Geoffrey E. Hinton,et al.  Recognizing Hand-written Digits Using Hierarchical Products of Experts , 2002, NIPS.

[27]  C. Cattani,et al.  On the Fractal Geometry of DNA by the Binary Image Analysis , 2013, Bulletin of mathematical biology.

[28]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[29]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[30]  H. Barlow.  Temporal and spatial summation in human vision at different background intensities , 1958, The Journal of physiology.

[31]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[32]  Xu Zhang,et al.  Spike-based indirect training of a spiking neural network-controlled virtual insect , 2013, 52nd IEEE Conference on Decision and Control.

[33]  R. Gray,et al.  Hippocampal synaptic transmission enhanced by low concentrations of nicotine , 1996, Nature.

[34]  E. Kandel.  The Molecular Biology of Memory Storage: A Dialogue Between Genes and Synapses , 2001, Science.

[35]  Thomas Schultz,et al.  An Artificial Brain Mechanism to Develop a Learning Paradigm for Robot Navigation , 2016 .

[36]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[37]  Yiran Chen,et al.  Forgetting memristor based neuromorphic system for pattern training and recognition , 2017, Neurocomputing.

[38]  Bernabé Linares-Barranco,et al.  On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex , 2011, Front. Neurosci..

[39]  R. Sarpeshkar,et al.  Brain power - borrowing from biology makes for low power computing [bionic ear] , 2006, IEEE Spectrum.

[40]  Gregory S. Snider,et al.  Spike-timing-dependent learning in memristive nanodevices , 2008, 2008 IEEE International Symposium on Nanoscale Architectures.

[41]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[42]  Richard Evans,et al.  Reinforcement Learning in a Neurally Controlled Robot Using Dopamine Modulated STDP , 2015, ArXiv.

[43]  Philip Goelet,et al.  The long and the short of long–term memory—a molecular framework , 1986, Nature.

[44]  Sungho Kim,et al.  Pattern Recognition Using Carbon Nanotube Synaptic Transistors with an Adjustable Weight Update Protocol. , 2017, ACS nano.

[45]  Jian Shi,et al.  A correlated nickelate synaptic transistor , 2013, Nature Communications.

[46]  Byung-Gook Park,et al.  Asymmetric dual-gate-structured one-transistor dynamic random access memory cells for retention characteristics improvement , 2016 .

[47]  J. Yang,et al.  Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. , 2017, Nature materials.

[48]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[49]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[50]  Jonathan R. Whitlock,et al.  Learning Induces Long-Term Potentiation in the Hippocampus , 2006, Science.

[51]  Farnood Merrikh-Bayat,et al.  Self-Adaptive Spike-Time-Dependent Plasticity of Metal-Oxide Memristors , 2015, Scientific Reports.

[52]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[53]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[54]  Yiran Chen,et al.  Memristor Crossbar-Based Neuromorphic Computing System: A Case Study , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[55]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[56]  Pinaki Mazumder,et al.  Memristor based STDP learning network for position detection , 2010, 2010 International Conference on Microelectronics.

[57]  Jongin Kim,et al.  Electronic system with memristive synapses for pattern recognition , 2015, Scientific Reports.

[58]  Yiran Chen,et al.  Reconfigurable Neuromorphic Computing System with Memristor-Based Synapse Design , 2013, Neural Processing Letters.

[59]  Alessandro Calderoni,et al.  Neuromorphic Learning and Recognition With One-Transistor-One-Resistor Synapses and Bistable Metal Oxide RRAM , 2016, IEEE Transactions on Electron Devices.

[60]  Rainer Waser,et al.  Complementary resistive switches for passive nanocrossbar memories. , 2010, Nature materials.

[61]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[62]  Taras Iakymchuk,et al.  Simplified spiking neural network architecture and STDP learning algorithm applied to image classification , 2015, EURASIP J. Image Video Process..

[63]  Byoung Hun Lee,et al.  Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron , 2015, IEEE Transactions on Industrial Electronics.

[64]  Tae Geun Kim,et al.  Substrate-Bias Assisted Hot Electron Injection Method for High-Speed, Low-Voltage, and Multi-Bit Flash Memories , 2011 .

[65]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[66]  Murray Shanahan,et al.  Training a spiking neural network to control a 4-DoF robotic arm based on Spike Timing-Dependent Plasticity , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[67]  Hani Hagras,et al.  Evolving spiking neural network controllers for autonomous robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[68]  Byung-Gook Park,et al.  Multi-threshold voltages in ultra thin-body devices by asymmetric dual-gate structure , 2015 .

[69]  Dario Floreano,et al.  From Wheels to Wings with Evolutionary Spiking Circuits , 2003, Artificial Life.

[70]  Byung-Gook Park,et al.  Neuromorphic System Based on CMOS Inverters and Si-Based Synaptic Device. , 2016, Journal of nanoscience and nanotechnology.

[71]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.