Memristor-CMOS Hybrid Circuit for Temporal-Pooling of Sensory and Hippocampal Responses of Cortical Neurons

As a software framework, Hierarchical Temporal Memory (HTM) has been developed to perform the brain’s neocortical functions, such as spatial and temporal pooling. However, it should be realized with hardware not software not only to mimic the neocortical function but also to exploit its architectural benefit. To do so, we propose a new memristor-CMOS (Complementary Metal-Oxide-Semiconductor) hybrid circuit of temporal-pooling here, which is composed of the input-layer and output-layer neurons mimicking the neocortex. In the hybrid circuit, the input-layer neurons have the proximal and basal/distal dendrites to combine sensory information with the temporal/location information from the brain’s hippocampus. Using the same crossbar architecture, the output-layer neurons can perform a prediction by integrating the temporal information on the basal/distal dendrites. For training the proposed circuit, we used only simple Hebbian learning, not the complicated backpropagation algorithm. Due to the simple hardware of Hebbian learning, the proposed hybrid circuit can be very suitable to online learning. The proposed memristor-CMOS hybrid circuit has been verified by the circuit simulation using the real memristor model. The proposed circuit has been verified to predict both the ordinal and out-of-order sequences. In addition, the proposed circuit has been tested with the external noise and memristance variation.

[1]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[2]  M. Stryker,et al.  Columnar Architecture Sculpted by GABA Circuits in Developing Cat Visual Cortex , 2004, Science.

[3]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[4]  L. Chua.  Memristor-The missing circuit element , 1971 .

[5]  H. Eichenbaum,et al.  Critical role of the hippocampus in memory for sequences of events , 2002, Nature Neuroscience.

[6]  Kyeong-Sik Min,et al.  New pulse amplitude modulation for fine tuning of memristor synapses , 2016, Microelectron. J..

[7]  Parami Wijesinghe,et al.  An All-Memristor Deep Spiking Neural Computing System: A Step Toward Realizing the Low-Power Stochastic Brain , 2017, IEEE Transactions on Emerging Topics in Computational Intelligence.

[8]  Yuwei Cui,et al.  The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding , 2016, bioRxiv.

[9]  Alex M. Thomson,et al.  Neocortical Layer 6, A Review , 2010, Front. Neuroanat..

[10]  Kyeong-Sik Min,et al.  Spatial-Pooling Memristor Crossbar Converting Sensory Information to Sparse Distributed Representation of Cortical Neurons , 2018, IEEE Transactions on Nanotechnology.

[11]  Karl J. Friston,et al.  The Functional Anatomy of Time: What and When in the Brain , 2016, Trends in Cognitive Sciences.

[12]  Kyeong-Sik Min,et al.  Dynamic Reference Scheme with Improved Read Voltage Margin for Compensating Cell-position and Background- pattern Dependencies in Pure Memristor Array , 2015 .

[13]  Seung Hwan Lee,et al.  Reservoir computing using dynamic memristors for temporal information processing , 2017, Nature Communications.

[14]  Daniel L Adams,et al.  The cortical column: a structure without a function , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[15]  Kyeong-Sik Min,et al.  New Twin Crossbar Architecture of Binary Memristors for Low-Power Image Recognition With Discrete Cosine Transform , 2015, IEEE Transactions on Nanotechnology.

[16]  Maciej Wielgosz,et al.  Formal Analysis of HTM Spatial Pooler Performance Under Predefined Operation Conditions , 2016, IJCRS.

[17]  Mirko Hansen,et al.  Unsupervised Hebbian learning experimentally realized with analogue memristive crossbar arrays , 2018, Scientific Reports.

[18]  R. Rosezin,et al.  High density 3D memory architecture based on the resistive switching effect , 2009 .

[19]  Leon O. Chua,et al.  Neuromemristive Circuits for Edge Computing: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Tony F. Wu,et al.  Monolithic 3D integration of logic and memory: Carbon nanotube FETs, resistive RAM, and silicon FETs , 2014, 2014 IEEE International Electron Devices Meeting.

[21]  Mirko Hansen,et al.  Memristive Hebbian Plasticity Model: Device Requirements for the Emulation of Hebbian Plasticity Based on Memristive Devices , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[22]  Gregory Cohen,et al.  EMNIST: an extension of MNIST to handwritten letters , 2017, CVPR 2017.

[23]  Subutai Ahmad,et al.  How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites , 2016, ArXiv.

[24]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[25]  Subutai Ahmad,et al.  Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory , 2015, ArXiv.

[26]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[27]  Alex Pappachen James,et al.  HTM Spatial Pooler With Memristor Crossbar Circuits for Sparse Biometric Recognition , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[28]  Kyeong-Sik Min,et al.  Sequential Memristor Crossbar for Neuromorphic Pattern Recognition , 2016, IEEE Transactions on Nanotechnology.

[29]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2015, Front. Neural Circuits.

[30]  Subutai Ahmad,et al.  Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[31]  Kevan A. C. Martin,et al.  A Canonical Microcircuit for Neocortex , 1989, Neural Computation.

[32]  Yuwei Cui,et al.  Continuous Online Sequence Learning with an Unsupervised Neural Network Model , 2015, Neural Computation.

[33]  Yuwei Cui,et al.  A Theory of How Columns in the Neocortex Enable Learning the Structure of the World , 2017, Front. Neural Circuits.

[34]  Emilio Kropff,et al.  Place cells, grid cells, and the brain's spatial representation system. , 2008, Annual review of neuroscience.

[35]  Seok-Jin Ham,et al.  Neuromorphic crossbar circuit with nanoscale filamentary-switching binary memristors for speech recognition , 2014, Nanoscale Research Letters.

[36]  S. Shipp.  Structure and function of the cerebral cortex , 2007, Current Biology.

[37]  Howard Eichenbaum,et al.  The hippocampus and memory for "what," "where," and "when". , 2004, Learning & memory.

[38]  Matthew Cook,et al.  Anatomical Constraints on Lateral Competition in Columnar Cortical Architectures , 2014, Neural Computation.

[39]  E. Rolls.  The hippocampus and memory , 1997 .

[40]  Alex Pappachen James,et al.  Hierarchical Temporal Memory Using Memristor Networks: A Survey , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.