Neuromorphic Implementation of Attractor Dynamics in a Two-Variable Winner-Take-All Circuit with NMDARs: A Simulation Study

Neural networks configured with winner-take-all (WTA) competition and N-methyl-D-aspartate receptor (NMDAR)-mediated synaptic dynamics are endowed with various dynamic characteristics of attractors underlying many cognitive functions. This paper presents a novel method for neuromorphic implementation of a two-variable WTA circuit with NMDARs aimed at implementing decision-making, working memory and hysteresis in visual perceptions. The method proposed is a dynamical system approach of circuit synthesis based on a biophysically plausible WTA model. Notably, slow and non-linear temporal dynamics of NMDAR-mediated synapses was generated. Circuit simulations in Cadence reproduced ramping neural activities observed in electrophysiological recordings in experiments of decision-making, the sustained activities observed in the prefrontal cortex during working memory, and classical hysteresis behavior during visual discrimination tasks. Furthermore, theoretical analysis of the dynamical system approach illuminated the underlying mechanisms of decision-making, memory capacity and hysteresis loops. The consistence between the circuit simulations and theoretical analysis demonstrated that the WTA circuit with NMDARs was able to capture the attractor dynamics underlying these cognitive functions. Their physical implementations as elementary modules are promising for assembly into integrated neuromorphic cognitive systems.

[1]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[2]  Xiao-Jing Wang.  Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[3]  Chiara Bartolozzi,et al.  Synaptic Dynamics in Analog VLSI , 2007, Neural Computation.

[4]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[5]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[6]  Chiara Bartolozzi,et al.  Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems , 2014, Proceedings of the IEEE.

[7]  P. Goldman-Rakic,et al.  Temporally irregular mnemonic persistent activity in prefrontal neurons of monkeys during a delayed response task. , 2003, Journal of neurophysiology.

[8]  Giacomo Indiveri,et al.  A device mismatch compensation method for VLSI neural networks , 2010, 2010 Biomedical Circuits and Systems Conference (BioCAS).

[9]  R. Sekuler,et al.  Hysteresis in the perception of motion direction as evidence for neural cooperativity , 1986, Nature.

[10]  Hongzhi You,et al.  Neuromorphic implementation of attractor dynamics in decision circuit with NMDARs , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[11]  Kazuyuki Aihara,et al.  Circuit Implementation and Dynamics of a Two-Dimensional MOSFET Neuron Model , 2007, Int. J. Bifurc. Chaos.

[12]  Daniel Robert,et al.  Synchrony through twice-frequency forcing for sensitive and selective auditory processing , 2009, Proceedings of the National Academy of Sciences.

[13]  Larissa Albantakis,et al.  The encoding of alternatives in multiple-choice decision-making , 2009, Proceedings of the National Academy of Sciences.

[14]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[15]  X. Wang,et al.  Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory , 1999, The Journal of Neuroscience.

[16]  Piotr Dudek,et al.  VLSI circuits implementing computational models of neocortical circuits , 2012, Journal of Neuroscience Methods.

[17]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[18]  Yulia Sandamirskaya,et al.  Dynamic neural fields as a step toward cognitive neuromorphic architectures , 2014, Front. Neurosci..

[19]  Timothy D. Hanks,et al.  Bounded Integration in Parietal Cortex Underlies Decisions Even When Viewing Duration Is Dictated by the Environment , 2008, The Journal of Neuroscience.

[20]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[21]  Yannick Bornat,et al.  A Library of Analog Operators Based on the Hodgkin-Huxley Formalism for the Design of Tunable, Real-Time, Silicon Neurons , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[22]  Patrick Camilleri,et al.  Robust Working Memory in an Asynchronously Spiking Neural Network Realized with Neuromorphic VLSI , 2011, Frontiers in Neuroscience.

[23]  Andrew Philippides,et al.  Unsupervised Learning in an Ensemble of Spiking Neural Networks Mediated by ITDP , 2016, PLoS Comput. Biol..

[24]  C. Koch,et al.  Attention activates winner-take-all competition among visual filters , 1999, Nature Neuroscience.

[25]  Sumio Hosaka,et al.  Associative memory realized by a reconfigurable memristive Hopfield neural network , 2015, Nature Communications.

[26]  Kwabena Boahen,et al.  Dynamical System Guided Mapping of Quantitative Neuronal Models Onto Neuromorphic Hardware , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Dante R. Chialvo,et al.  Modulated noisy biological dynamics: Three examples , 1993 .

[28]  Massimiliano Giulioni,et al.  Decision making and perceptual bistability in spike-based neuromorphic VLSI systems , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[29]  Kwabena Boahen,et al.  Silicon-Neuron Design: A Dynamical Systems Approach , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[30]  J. Fellous,et al.  A role for NMDA-receptor channels in working memory , 1998, Nature Neuroscience.

[31]  H. Haken.  Synergetics: an Introduction, Nonequilibrium Phase Transitions and Self-organization in Physics, Chemistry, and Biology , 1977 .

[32]  Jia Wang,et al.  DaDianNao: A Machine-Learning Supercomputer , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[33]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[34]  Kazuyuki Aihara,et al.  A two-variable silicon neuron circuit based on the Izhikevich model , 2011, Artificial Life and Robotics.

[35]  Gert Cauwenberghs,et al.  Analog VLSI Biophysical Neurons and Synapses With Programmable Membrane Channel Kinetics , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[36]  Giacomo Indiveri,et al.  Synthesizing cognition in neuromorphic electronic systems , 2013, Proceedings of the National Academy of Sciences.

[37]  Davide Badoni,et al.  Spike-Driven Synaptic Plasticity: Theory, Simulation, VLSI Implementation , 2000, Neural Computation.

[38]  Arindam Basu,et al.  Analysis and reduction of mismatch in silicon neurons , 2011, 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[39]  KongFatt Wong-Lin,et al.  Neural Circuit Dynamics Underlying Accumulation of Time-Varying Evidence During Perceptual Decision Making , 2007, Frontiers Comput. Neurosci..

[40]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[41]  Emmanuel M. Drakakis,et al.  Systematic Computation of Nonlinear Cellular and Molecular Dynamics with Low-Power CytoMimetic Circuits: A Simulation Study , 2013, PloS one.

[42]  H. Haken,et al.  Synergetics , 1988, IEEE Circuits and Devices Magazine.

[43]  Karl J. Friston,et al.  The Neural Structures Expressing Perceptual Hysteresis in Visual Letter Recognition , 2002, Neuron.

[44]  Giacomo Indiveri,et al.  Frontiers in Neuromorphic Engineering , 2011, Front. Neurosci..

[45]  P. Holmes,et al.  The dynamics of choice among multiple alternatives , 2006 .

[46]  E. Knudsen.  Fundamental components of attention. , 2007, Annual review of neuroscience.

[47]  T. Sejnowski,et al.  Neurocomputational models of working memory , 2000, Nature Neuroscience.

[48]  M. Shadlen,et al.  Decision Making as a Window on Cognition , 2013, Neuron.

[49]  Hongzhi You,et al.  The neural dynamics for hysteresis in visual perception , 2011, Neurocomputing.

[50]  G. E. Alexander,et al.  Neuron Activity Related to Short-Term Memory , 1971, Science.

[51]  Hongzhi You,et al.  Dynamics of Multiple-Choice Decision Making , 2013, Neural Computation.

[52]  Tianshi Chen,et al.  DaDianNao: A Neural Network Supercomputer , 2017, IEEE Transactions on Computers.

[53]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[54]  Andrew Nere,et al.  A Neuromorphic Architecture for Object Recognition and Motion Anticipation Using Burst-STDP , 2012, PloS one.

[55]  Giacomo Indiveri,et al.  Spike-based learning with a generalized integrate and fire silicon neuron , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[56]  B. Gilbert.  Translinear circuits: a proposed classification , 1975 .

[57]  R. Douglas,et al.  Event-Based Neuromorphic Systems , 2015 .

[58]  Xiao-Jing Wang,et al.  From Distributed Resources to Limited Slots in Multiple-Item Working Memory: A Spiking Network Model with Normalization , 2012, The Journal of Neuroscience.