A neurocomputational model of classical conditioning phenomena: A putative role for the hippocampal region in associative learning

Some existing models of hippocampal function simulate performance in classical conditioning tasks using the error backpropagation algorithm to guide learning (Gluck, M.A., and Myers, C.E., (1993). Hippocampal mediation of stimulus representation: a computational theory. Hippocampus, 3(4), 491-516.). This algorithm is not biologically plausible because it requires information to be passed backward through layers of nodes and assumes that the environment provides information to the brain about what correct outputs should be. Here, we show that the same information-processing function proposed for the hippocampal region in the Gluck and Myers (1993) model can also be implemented in a network without using the backpropagation algorithm. Instead, our newer instantiation of the theory uses only (a) Hebbian learning methods which match more closely with synaptic and associative learning mechanisms ascribed to the hippocampal region and (b) a more plausible representation of input stimuli. We demonstrate here that this new more biologically plausible model is able to simulate various behavioral effects, including latent inhibition, acquired equivalence, sensory preconditioning, negative patterning, and context shift effects. In addition, the newer model is able to address some new phenomena including the effect of the number of training trials on blocking and overshadowing.

[1]  M. Hasselmo,et al.  Computational Modeling of Entorhinal Cortex , 2000, Annals of the New York Academy of Sciences.

[2]  Catherine E. Myers,et al.  Further implications of a computational model of septohippocampal cholinergic modulation in eyeblink conditioning , 1998, Psychobiology.

[3]  R. Lubow Latent inhibition. , 1973, Psychological bulletin.

[4]  Geoffrey Hall,et al.  Acquired equivalence of cues in pigeon autoshaping: Effects of training with common consequences and with common antecedents , 1993 .

[5]  Michael E. Hasselmo,et al.  Hippocampal Mechanisms for the Context-dependent Retrieval of Episodes Action Reward Place (state) , 2022 .

[6]  H. Eichenbaum,et al.  Memory Representation within the Parahippocampal Region , 1997, The Journal of Neuroscience.

[7]  Joseph E. Steinmetz,et al.  Cerebellar cortical lesions and reacquisition in classical conditioning of the nictitating membrane response in rabbits , 1993, Brain Research.

[8]  M. Hasselmo,et al.  Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function , 1997, Behavioural Brain Research.

[9]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[10]  Bruce L McNaughton,et al.  Selective excitotoxic lesions of the hippocampus and basolateral amygdala have dissociable effects on appetitive cue and place conditioning based on path integration in a novel Y‐maze procedure , 2006, The European journal of neuroscience.

[11]  Matt L. Huffman,et al.  When More is Less , 1997 .

[12]  Howard Eichenbaum,et al.  How does the hippocampus contribute to memory? , 2003, Trends in Cognitive Sciences.

[13]  Marc W Howard,et al.  Modeling of context-dependent retrieval in hippocampal region CA1: Implications for cognitive function in schizophrenia , 2007, Schizophrenia Research.

[14]  M. Gluck,et al.  Dissociating Hippocampal versus Basal Ganglia Contributions to Learning and Transfer , 2003, Journal of Cognitive Neuroscience.

[15]  Frank Puga,et al.  Functional networks underlying latent inhibition learning in the mouse brain , 2007, NeuroImage.

[16]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[17]  Catherine E. Myers,et al.  Selective hippocampal lesions disrupt a novel cue effect but fail to eliminate blocking in rabbit eyeblink conditioning , 2002, Cognitive, affective & behavioral neuroscience.

[18]  Gwenn S. Smith,et al.  EARLY MARKER FOR ALZHEIMER'S DISEASE: THE ATROPHIC HIPPOCAMPUS , 1989, The Lancet.

[19]  T. Robbins,et al.  The hippocampus and appetitive Pavlovian conditioning: Effects of excitotoxic hippocampal lesions on conditioned locomotor activity and autoshaping , 2005, Hippocampus.

[20]  M. Hasselmo,et al.  Cholinergic Deafferentation of the Entorhinal Cortex in Rats Impairs Encoding of Novel But Not Familiar Stimuli in a Delayed Nonmatch-to-Sample Task , 2005, The Journal of Neuroscience.

[21]  Catherine E. Myers,et al.  Cerebellar Substrates for Error Correction in Motor Conditioning , 2001, Neurobiology of Learning and Memory.

[22]  D. Nicholson,et al.  Lesions of the perirhinal cortex impair sensory preconditioning in rats , 2000, Behavioural Brain Research.

[23]  R. F. Thompson,et al.  Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. , 1986, Behavioral neuroscience.

[24]  D. Kumaran,et al.  The Dynamics of Hippocampal Activation during Encoding of Overlapping Sequences , 2006, Neuron.

[25]  R. C. Honey,et al.  Acquired equivalence and distinctiveness of cues: II. Neural manipulations and their implications. , 2002, Journal of experimental psychology. Animal behavior processes.

[26]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[27]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[28]  D. Davies,et al.  MYOCARDIAL DISORDERS IN THE ELDERLY. , 1963, Lancet.

[29]  G. Hall,et al.  Acquired equivalence between cues trained with a common antecedent. , 1993, Journal of experimental psychology. Animal behavior processes.

[30]  R. F. Thompson,et al.  Neuronal substrate of classical conditioning in the hippocampus , 1976, Science.

[31]  M. Eacott,et al.  The Roles of Perirhinal Cortex, Postrhinal Cortex, and the Fornix in Memory for Objects, Contexts, and Events in the Rat , 2005, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[32]  R. Richardson,et al.  Latent inhibition in the developing rat: an examination of context-specific effects. , 2005, Developmental psychobiology.

[33]  J. Gray,et al.  Increased extracellular dopamine in the nucleus accumbens of the rat during associative learning of neutral stimuli , 1998, Neuroscience.

[34]  Richard F. Thompson,et al.  Effects of lidocaine injection in the interpositus nucleus and red nucleus on conditioned behavioral and neuronal responses , 1990, Brain Research.

[35]  Li Yang,et al.  Involvement of the ipsilateral and contralateral cerebellum in the acquisition of unilateral classical eyeblink conditioning in guinea pigs , 2009, Acta Pharmacologica Sinica.

[36]  L. Kamin Predictability, surprise, attention, and conditioning , 1967 .

[37]  Richard F. Thompson,et al.  Topics in learning and performance , 1972 .

[38]  S. Engel,et al.  Dynamics of the Hippocampus During Encoding and Retrieval of Face-Name Pairs , 2003, Science.

[39]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[40]  Michael E. Hasselmo,et al.  A Proposed Function for Hippocampal Theta Rhythm: Separate Phases of Encoding and Retrieval Enhance Reversal of Prior Learning , 2002, Neural Computation.

[41]  N. Schmajuk,et al.  Stimulus configuration, classical conditioning, and hippocampal function. , 1992, Psychological review.

[42]  D. Bilkey Long-term potentiation in the in vitro perirhinal cortex displays associative properties , 1996, Brain Research.

[43]  J F Disterhoft,et al.  Hippocampal encoding of non‐spatial trace conditioning , 1999, Hippocampus.

[44]  C. Myers,et al.  Selectively Impaired Associative Learning in Older People with Cognitive Decline , 2002, Journal of Cognitive Neuroscience.

[45]  E. Macphail,et al.  Hippocampal Lesions in Pigeons (Columba Livia) Disrupt Reinforced Preexposure but not Overshadowing or Blocking , 1994, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[46]  S. Becker,et al.  A neural network model of hippocampal–striatal–prefrontal interactions in contextual conditioning , 2008, Brain Research.

[47]  C. C. Spiker Experiments with children on the hypotheses of acquired distinctiveness and equivalence of cues. , 1956, Child development.

[48]  N. Mackintosh,et al.  Successful overshadowing and blocking in hippocampectomized rats , 1984, Behavioural Brain Research.

[49]  P. Solomon Role of the hippocampus in blocking and conditioned inhibition of the rabbit's nictitating membrane response. , 1977, Journal of comparative and physiological psychology.

[50]  D. Paré,et al.  The rhinal cortices: a wall of inhibition between the neocortex and the hippocampus , 2004, Progress in Neurobiology.

[51]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[52]  R. Suri Anticipatory responses of dopamine neurons and cortical neurons reproduced by internal model , 2001, Experimental Brain Research.

[53]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[54]  F. Gregory Ashby,et al.  FROST: A Distributed Neurocomputational Model of Working Memory Maintenance , 2005, Journal of Cognitive Neuroscience.

[55]  H Eichenbaum,et al.  Selective damage to the hippocampal region blocks long‐term retention of a natural and nonspatial stimulus‐stimulus association , 1995, Hippocampus.

[56]  A. Cándido,et al.  Dorsal hippocampal lesions impair blocking but not latent inhibition of taste aversion learning in rats. , 1995, Behavioral neuroscience.

[57]  R G Phillips,et al.  Contributions of postrhinal and perirhinal cortex to contextual information processing. , 2000, Behavioral neuroscience.

[58]  M. Gluck,et al.  Interactive memory systems in the human brain , 2001, Nature.

[59]  Lee Ryan,et al.  The effect of scene context on episodic object recognition: Parahippocampal cortex mediates memory encoding and retrieval success , 2007, Hippocampus.

[60]  P. Solomon,et al.  Latent inhibition and stimulus generalization of the classically conditioned nictitating membrane response in rabbits (Oryctolagus cuniculus) following dorsal hippocampal ablation. , 1975, Journal of comparative and physiological psychology.

[61]  Catherine E. Myers,et al.  Associative learning in deficit and nondeficit schizophrenia , 2008, Neuroreport.

[62]  M. Gluck,et al.  A comparison of latent inhibition and learned irrelevance pre-exposure effects in rabbit and human eyeblink conditioning , 2002, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[63]  M. Gluck,et al.  Dissociating medial temporal and basal ganglia memory systems with a latent learning task , 2003, Neuropsychologia.

[64]  M. Hasselmo,et al.  A Computational Model of Cholinergic Disruption of Septohippocampal Activity in Classical Eyeblink Conditioning , 1996, Neurobiology of Learning and Memory.

[65]  M. M. Patterson,et al.  Fimbrial lesions and sensory preconditioning. , 1984, Behavioral neuroscience.

[66]  M. M. Patterson,et al.  Differential effects of hippocampectomy on classically conditioned rabbit nictitating membrane response related to interstimulus interval. , 1985, Behavioral neuroscience.

[67]  P. Holland,et al.  Effects of hippocampal lesions in overshadowing and blocking procedures. , 2003, Behavioral neuroscience.

[68]  M. Gluck,et al.  Integrating incremental learning and episodic memory models of the hippocampal region. , 2005, Psychological review.

[69]  T. Robbins,et al.  Complementary roles for the amygdala and hippocampus in aversive conditioning to explicit and contextual cues , 1991, Neuroscience.

[70]  Anthony S. Maida,et al.  Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia , 2007, Cognitive Systems Research.

[71]  H. Eichenbaum,et al.  Hippocampal system dysfunction and odor discrimination learning in rats: impairment or facilitation depending on representational demands. , 1988, Behavioral neuroscience.

[72]  M. Gluck,et al.  Hippocampal mediation of stimulus representation: A computational theory , 1993, Hippocampus.

[73]  M. Gluck,et al.  Dissociating entorhinal and hippocampal involvement in latent inhibition. , 2000, Behavioral neuroscience.

[74]  M. Gluck,et al.  Dissociation of hippocampal and entorhinal function in associative learning: A computational approach , 1995, Psychobiology.

[75]  R. E. Lubow,et al.  The Context Effect: The Relationship Between Stimulus Preexposure and Environmental Preexposure Determines Subsequent Learning. , 1976 .

[76]  G. Cicala,et al.  Increased conditioning in rats to a blocked CS after the first compound trial , 1988 .

[77]  H. Terrace,et al.  OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR DISCRIMINATION LEARNING WITH AND WITHOUT " ERRORS " ' , 2005 .

[78]  Francisco Arcediano,et al.  Overshadowing as a function of trial number: Dynamics of first- and second-order comparator effects , 2003, Learning & behavior.

[79]  R. O’Reilly,et al.  Hippocampal formation supports conditioning to memory of a context. , 2002, Behavioral neuroscience.

[80]  G. Winocur,et al.  The cognitive neuroscience of remote episodic, semantic and spatial memory , 2006, Current Opinion in Neurobiology.

[81]  R. C. Honey,et al.  Context-specific conditioning in the conditioned-emotional-response procedure. , 1990, Journal of experimental psychology. Animal behavior processes.

[82]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[83]  R. O’Reilly,et al.  Contextual fear conditioning, conjunctive representations, pattern completion, and the hippocampus. , 1999, Behavioral neuroscience.

[84]  M. Gluck,et al.  Computational models of the hippocampal region: linking incremental learning and episodic memory , 2003, Trends in Cognitive Sciences.

[85]  G. Bower,et al.  From conditioning to category learning: an adaptive network model. , 1988, Journal of experimental psychology. General.

[86]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[87]  J. Theios,et al.  Acquisition and extinction of a classically conditioned response in hippocampectomized rabbits (Oryctolagus cuniculus). , 1972, Journal of comparative and physiological psychology.

[88]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[89]  B. Weber,et al.  Human hippocampus establishes associations in memory , 1997, Hippocampus.

[90]  H. Eichenbaum,et al.  The hippocampus and memory for orderly stimulus relations. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[91]  G. Hall,et al.  The role of mediated conditioning in acquired equivalence. , 1999, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[92]  Joseph E LeDoux,et al.  Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. , 1992, Behavioral neuroscience.

[93]  Ivo D Dinov,et al.  3D comparison of hippocampal atrophy in amnestic mild cognitive impairment and Alzheimer's disease. , 2006, Brain : a journal of neurology.

[94]  A. Barto Adaptive Critics and the Basal Ganglia , 1995 .

[95]  R. O’Reilly,et al.  Opinion TRENDS in Cognitive Sciences Vol.6 No.12 December 2002 , 2022 .

[96]  H. Shill,et al.  Motor learning in essential tremor , 2009, Movement disorders : official journal of the Movement Disorder Society.

[97]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[98]  Jae-Wook Ryou,et al.  Lesions of the Entorhinal Cortex Impair Acquisition of Hippocampal-Dependent Trace Conditioning , 2001, Neurobiology of Learning and Memory.

[99]  P. Solomon,et al.  Altered activity in the hippocampus is more detrimental to classical conditioning than removing the structure. , 1983, Science.

[100]  G. Hall,et al.  Context Specificity of Latent Inhibition in Taste Aversion Learning , 1986, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[101]  Oskar Pineño,et al.  When more is less: Extending training of the blocking association following compound training attenuates the blocking effect , 2006, Learning & behavior.

[102]  J. Disterhoft,et al.  Sequence of single neuron changes in CA1 hippocampus of rabbits during acquisition of trace eyeblink conditioned responses. , 1997, Journal of neurophysiology.

[103]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.