Improving Associative Memory in a Network of Spiking Neurons

Associative neural network models are a commonly used methodology when investigating the theory of associative memory in the brain. Comparisons between the mammalian hippocampus and neural network models of associative memory have been investigated [7]. Biologically based networks are complex systems built of neurons with a variety of properties. Here we compare and contrast associative memory function in a network of biologically-based spiking neurons [14] with previously published results for a simple artificial neural network model [6]. We investigate biologically plausible implementations of methods for improving recall under biologically realistic conditions, such as a sparsely connected network.

[1]  T. Gillessen,et al.  Amplification of EPSPs by low Ni(2+)- and amiloride-sensitive Ca2+ channels in apical dendrites of rat CA1 pyramidal neurons. , 1997, Journal of neurophysiology.

[2]  Günther Palm,et al.  Iterative Retrieval In Associative Memories By Threshold Control Of Different Neural Models , 1995 .

[3]  Alessandro Treves,et al.  Uninformative memories will prevail: The storage of correlated representations and its consequences , 2007, HFSP journal.

[4]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[5]  J. Bower,et al.  Cortical oscillations and temporal interactions in a computer simulation of piriform cortex. , 1992, Journal of neurophysiology.

[6]  Pablo Fuentealba,et al.  Cell Type-Specific Tuning of Hippocampal Interneuron Firing during Gamma Oscillations In Vivo , 2007, The Journal of Neuroscience.

[7]  Simon M Stringer,et al.  Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning , 2006, Network.

[8]  Michael E. Hasselmo,et al.  Acetylcholine and Learning in a Cortical Associative Memory , 1993, Neural Computation.

[9]  Marco Idiart,et al.  Memory retrieval time and memory capacity of the CA3 network: role of gamma frequency oscillations. , 2007, Learning & memory.

[10]  John Rinzel,et al.  Intrinsic and network rhythmogenesis in a reduced traub model for CA3 neurons , 1995, Journal of Computational Neuroscience.

[11]  G. Buzsáki,et al.  Interneurons of the hippocampus , 1998, Hippocampus.

[12]  A. Burgess,et al.  Functional connectivity of gamma EEG activity is modulated at low frequency during conscious recollection. , 2002, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  M. L. Lambon Ralph,et al.  Prototypicality, distinctiveness, and intercorrelation: Analyses of the semantic attributes of living and nonliving concepts , 2001, Cognitive neuropsychology.

[14]  Wulfram Gerstner,et al.  Associative memory in a network of ‘spiking’ neurons , 1992 .

[15]  Ole Paulsen,et al.  Hippocampal gamma‐frequency oscillations: from interneurones to pyramidal cells, and back , 2005, The Journal of physiology.

[16]  D. Norman.  Learning and Memory , 1982 .

[17]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[18]  J. Magee,et al.  Somatic EPSP amplitude is independent of synapse location in hippocampal pyramidal neurons , 2000, Nature Neuroscience.

[19]  Naoyuki Sato,et al.  Memory Encoding by Theta Phase Precession in the Hippocampal Network , 2003, Neural Computation.

[20]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[21]  György Buzsáki,et al.  Hippocampal place cell assemblies are speed-controlled oscillators , 2007, Proceedings of the National Academy of Sciences.

[22]  Sacha B. Nelson,et al.  Synaptic depression: a key player in the cortical balancing act , 1998, Nature Neuroscience.

[23]  Xiaojing Wang,et al.  Analysis of complex bursting in cortical pyramidal neuron models , 2000, Neurocomputing.

[24]  Susan L. Wearne,et al.  Neuronal Firing Sensitivity to Morphologic and Active Membrane Parameters , 2007, PLoS Comput. Biol..

[25]  Ali A. Minai,et al.  Latent Attractors: A Model for Context-Dependent Place Representations in the Hippocampus , 2000, Neural Computation.

[26]  Bernhard Hellwig,et al.  A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex , 2000, Biological Cybernetics.

[27]  J. Lisman,et al.  Physiologically realistic formation of autoassociative memory in networks with theta/gamma oscillations: role of fast NMDA channels. , 1996, Learning & memory.

[28]  David Willshaw,et al.  On setting unit thresholds in an incompletely connected associative net Network: Comput , 1993 .

[29]  E. Rolls,et al.  A computational theory of hippocampal function, and empirical tests of the theory , 2006, Progress in Neurobiology.

[30]  R. Traub,et al.  A model of a CA3 hippocampal pyramidal neuron incorporating voltage-clamp data on intrinsic conductances. , 1991, Journal of neurophysiology.

[31]  D. Johnston,et al.  Synaptic activation of voltage-gated channels in the dendrites of hippocampal pyramidal neurons. , 1995, Science.

[32]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[33]  E. Rolls.  Principles underlying the representation and storage of information in neuronal networks in the primate hippocampus and cerebral cortex , 1990 .

[34]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[35]  G. Buzsáki.  Two-stage model of memory trace formation: A role for “noisy” brain states , 1989, Neuroscience.

[36]  A. I. Isakov.  Project progress report , 1997 .

[37]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[38]  Xiao-Jing Wang,et al.  Bursting Neurons Signal Input Slope , 2002, The Journal of Neuroscience.

[39]  L. M. Ward,et al.  Synchronous neural oscillations and cognitive processes , 2003, Trends in Cognitive Sciences.

[40]  Hatsuo Hayashi,et al.  Stochastic resonance in the hippocampal CA3-CA1 model: a possible memory recall mechanism , 2002, Neural Networks.

[41]  Thomas Wennekers,et al.  On the relation between neural modelling and experimental neuroscience , 1997 .

[42]  G. Buzsáki,et al.  Temporal structure in spatially organized neuronal ensembles: a role for interneuronal networks , 1995, Current Opinion in Neurobiology.

[43]  M. Hasselmo,et al.  Modulation of the input/output function of rat piriform cortex pyramidal cells. , 1994, Journal of neurophysiology.

[44]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[45]  B. McNaughton,et al.  Hippocampal synaptic enhancement and information storage within a distributed memory system , 1987, Trends in Neurosciences.

[46]  E. Rolls.  The representation and storage of information in neural networks in the primate cerebral cortex and hippocampus , 1989 .

[47]  E Fransén,et al.  A model of cortical associative memory based on a horizontal network of connected columns. , 1998, Network.

[48]  Chris McNorgan,et al.  An attractor model of lexical conceptual processing: simulating semantic priming , 1999, Cogn. Sci..

[49]  J E Lisman,et al.  Storage of 7 +/- 2 short-term memories in oscillatory subcycles , 1995, Science.

[50]  B. Connors,et al.  Intrinsic firing patterns of diverse neocortical neurons , 1990, Trends in Neurosciences.

[51]  Leif H. Finkel,et al.  Neuromodulatory control of hippocampal function: towards a model of Alzheimer's disease , 1998, Artif. Intell. Medicine.

[52]  David J. Willshaw,et al.  Improving recall from an associative memory , 1995, Biological Cybernetics.

[53]  Mark S. Seidenberg,et al.  On the nature and scope of featural representations of word meaning. , 1997, Journal of experimental psychology. General.

[54]  Edmund T. Rolls,et al.  The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain , 1990 .

[55]  R. Lipowsky,et al.  Dendritic Na+ channels amplify EPSPs in hippocampal CA1 pyramidal cells. , 1996, Journal of neurophysiology.

[56]  B. Connors,et al.  Bypassing interneurons: inhibition in neocortex , 2007, Nature Neuroscience.

[57]  C. Elger,et al.  Human memory formation is accompanied by rhinal–hippocampal coupling and decoupling , 2001, Nature Neuroscience.

[58]  D. Amaral,et al.  Organization of intrahippocampal projections originating from CA3 pyramidal cells in the rat , 1990, The Journal of comparative neurology.

[59]  T. Bliss,et al.  The Hippocampus Book , 2006 .

[60]  A. Lansner,et al.  Modelling Hebbian cell assemblies comprised of cortical neurons , 1992 .

[61]  P. Jonas,et al.  Synaptic mechanisms of synchronized gamma oscillations in inhibitory interneuron networks , 2007, Nature Reviews Neuroscience.

[62]  C. Stevens,et al.  Changes in reliability of synaptic function as a mechanism for plasticity , 1994, Nature.

[63]  E. Taub,et al.  Effects of constraint-induced movement therapy on patients with chronic motor deficits after stroke: a replication. , 1999, Stroke.

[64]  Eytan Domany,et al.  Models of Neural Networks I , 1991 .

[65]  Ivan Soltesz,et al.  Role of mossy fiber sprouting and mossy cell loss in hyperexcitability: a network model of the dentate gyrus incorporating cell types and axonal topography. , 2005, Journal of neurophysiology.

[66]  D. Lewis,et al.  Cortical inhibitory neurons and schizophrenia , 2005, Nature Reviews Neuroscience.

[67]  Jens Markert,et al.  Neural associative memories for language understanding and robot control , 2009 .

[68]  George S. Cree,et al.  Distinctive features hold a privileged status in the computation of word meaning: Implications for theories of semantic memory. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[69]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[70]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[71]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[72]  E. Rolls.  Functions of neuronal networks in the hippocampus and neocortex in memory , 1989 .

[73]  M. Frotscher,et al.  Rapid Signaling at Inhibitory Synapses in a Dentate Gyrus Interneuron Network , 2001, The Journal of Neuroscience.

[74]  Günther Palm,et al.  Coexistence of Cell Assemblies and STDP , 2009, ICANN.

[75]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[76]  G. Buzsáki,et al.  Gamma (40-100 Hz) oscillation in the hippocampus of the behaving rat , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[77]  O Jensen,et al.  Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory: role of NMDA channels in recall. , 1996, Learning & memory.

[78]  Philippe De Wilde.  Neural Network Models , 1996 .

[79]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[80]  E. Rolls,et al.  Computational analysis of the role of the hippocampus in memory , 1994, Hippocampus.

[81]  A. R. Gardner-Medwin.  Doubly modifiable synapses: a model of short and long term auto-associative memory , 1989, Proceedings of the Royal Society of London. B. Biological Sciences.

[82]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[83]  E. Halgren,et al.  Role of inhibition in memory retrieval by hippocampal area CA3 , 1994, Neuroscience & Biobehavioral Reviews.

[84]  T. Sejnowski,et al.  Neurocomputational models of working memory , 2000, Nature Neuroscience.

[85]  Bruce P. Graham,et al.  Pattern recognition in a compartmental model of a CA1 pyramidal neuron , 2001 .

[86]  B. Sakmann,et al.  Calcium electrogenesis in distal apical dendrites of layer 5 pyramidal cells at a critical frequency of back-propagating action potentials. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[87]  Peter Dayan,et al.  Rate- and Phase-coded Autoassociative Memory , 2004, NIPS.

[88]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[89]  M. Hasselmo,et al.  Cholinergic modulation of cortical associative memory function. , 1992, Journal of neurophysiology.

[90]  G. Buzsáki,et al.  Restoration of rhythmic slow activity (θ) in the subcortically denervated hippocampus by fetal CNS transplants , 1987, Brain Research.

[91]  Peter J. B. Hancock,et al.  Preprocessing images of faces: correlations with human perceptions of distinctiveness and familiarity , 1995 .

[92]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[93]  Murat Okatan,et al.  Frequency-dependent synaptic potentiation, depression and spike timing induced by Hebbian pairing in cortical pyramidal neurons , 2000, Neural Networks.

[94]  J. Bower,et al.  Simulated responses of cerebellar Purkinje cells are independent of the dendritic location of granule cell synaptic inputs. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[95]  Bruce P. Graham,et al.  Improving Recall in an Associative Neural Network of Spiking Neurons , 2007, Summer School on Neural Networks.

[96]  B L McNaughton,et al.  Dynamics of the hippocampal ensemble code for space. , 1993, Science.

[97]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[98]  Idan Segev,et al.  Methods in neuronal modeling: From synapses to networks , 1989 .

[99]  Friedrich T. Sommer,et al.  Associative Data Storage and Retrieval in Neural Networks , 1996 .

[100]  David Willshaw,et al.  Capacity and information efficiency of the associative net , 1997 .

[101]  S. Raghavachari,et al.  Distinct patterns of brain oscillations underlie two basic parameters of human maze learning. , 2001, Journal of neurophysiology.

[102]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[103]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[104]  Bartlett W. Mel,et al.  Arithmetic of Subthreshold Synaptic Summation in a Model CA1 Pyramidal Cell , 2003, Neuron.

[105]  Richard Durbin,et al.  The computing neuron , 1989 .

[106]  Thomas Wennekers,et al.  Modelling studies on the computational function of fast temporal structure in cortical circuit activity , 2000, Journal of Physiology-Paris.

[107]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[108]  J. Lisman.  Relating Hippocampal Circuitry to Function Recall of Memory Sequences by Reciprocal Dentate–CA3 Interactions , 1999, Neuron.

[109]  G. Buzsáki,et al.  Intracellular correlates of hippocampal theta rhythm in identified pyramidal cells, granule cells, and basket cells , 1995, Hippocampus.

[110]  S. Siegelbaum,et al.  Regulation of hippocampal transmitter release during development and long-term potentiation. , 1995, Science.

[111]  Thomas Wennekers,et al.  Associative memory in networks of spiking neurons , 2001, Neural Networks.

[112]  P. Somogyi,et al.  Synchronization of neuronal activity in hippocampus by individual GABAergic interneurons , 1995, Nature.

[113]  Adam Kepecs,et al.  Information encoding and computation with spikes and bursts , 2003, Network.

[114]  Michael L. Hines,et al.  The NEURON Book , 2006 .

[115]  H. Markram,et al.  The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. , 1997, Proceedings of the National Academy of Sciences of the United States of America.