Neural Network Simulations of a Possible Role of the Hippocampus in Pavlovian Conditioning

In this paper, we simulate the effects of hippocampal lesions on Pavlovian conditioning with an existing neural network model. According to the model, the hippocampus sends a diffuse discrepancy signal that modulates efficacies of synapses from primary sensory to polysensory areas. We hypothesize that this signal lessens the detrimental effects of momentary nonreinforcement and weaker cues on such efficacies in Pavlovian conditioning. Hippocampal lesions are thus hypothesized to exacerbate both detrimental effects. To test this hypothesis against some relevant animal evidence, we ran two computer simulations using simple feedforward neural networks with two hidden layers, designed according to the model. Hippocampal lesions were simulated by removing the networks’ hippocampal units. Networks were trained in various conditions involving momentary nonreinforcement and reinforcement of differently salient cues. The results were reasonably consistent with animal evidence that hippocampal lesion is more disruptive of long-delay than short-trace conditioning (Simulation 1) and backward more than contiguous-trace conditioning (Simulation 2). Implications, limitations, and future directions are discussed.

[1]  Ralph R. Miller,et al.  Conditioned suppression is an inverted-U function of footshock intensity , 2013, Learning & behavior.

[2]  J. Theios,et al.  Acquisition and extinction of a classically conditioned response in hippocampectomized rabbits (Oryctolagus cuniculus). , 1972, Journal of comparative and physiological psychology.

[3]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[4]  J. Donahoe,et al.  Behavior analysis and revaluation. , 2000, Journal of the experimental analysis of behavior.

[5]  T. L. Bennett,et al.  Hippocampectomy and the attenuation of blocking. , 1978, Behavioral biology.

[6]  R. F. Thompson,et al.  Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. , 1986, Behavioral neuroscience.

[7]  D. Amaral,et al.  Cortical afferents of the perirhinal, postrhinal, and entorhinal cortices of the rat , 1998 .

[8]  Herve Simon,et al.  Origin of dopaminergic innervation of the rat hippocampal formation , 1980, Neuroscience Letters.

[9]  J W Donahoe,et al.  A selectionist approach to reinforcement. , 1993, Journal of the experimental analysis of behavior.

[10]  L. Squire,et al.  Human amnesia and the medial temporal region: enduring memory impairment following a bilateral lesion limited to field CA1 of the hippocampus , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  G. V. Van Hoesen,et al.  Hippocampal formation: anatomy and the patterns of pathology in Alzheimer's disease. , 1990, Progress in brain research.

[12]  U. Frey,et al.  Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices , 1990, Brain Research.

[13]  M. M. Patterson,et al.  Differential effects of hippocampectomy on classically conditioned rabbit nictitating membrane response related to interstimulus interval. , 1985, Behavioral neuroscience.

[14]  P. Holland,et al.  Effects of hippocampal lesions in overshadowing and blocking procedures. , 2003, Behavioral neuroscience.

[15]  A. Blaisdell,et al.  Backward conditioning: mediation by the context. , 2003, Journal of experimental psychology. Animal behavior processes.

[16]  M. Treviño,et al.  Associative Learning Through Acquired Salience , 2016, Front. Behav. Neurosci..

[17]  M. Gluck,et al.  Hippocampal mediation of stimulus representation: A computational theory , 1993, Hippocampus.

[18]  N. Schmajuk,et al.  A neural network approach to hippocampal function in classical conditioning. , 1991, Behavioral neuroscience.

[19]  J. Disterhoft,et al.  Hippocampectomy disrupts trace eye-blink conditioning in rabbits. , 1990, Behavioral neuroscience.

[20]  N. A. Marlin Contextual associations in trace conditioning , 1981 .

[21]  J. Pearce,et al.  Hippocampal lesions attenuate latent inhibition of a CS and of a neutral stimulus , 1987, Psychobiology.

[22]  M. Fanselow,et al.  Post‐training excitotoxic lesions of the dorsal hippocampus attenuate forward trace, backward trace, and delay fear conditioning in a temporally specific manner , 2002, Hippocampus.

[23]  D. Amaral,et al.  Evidence for a direct projection from the superior temporal gyrus to the entorhinal cortex in the monkey , 1983, Brain Research.

[24]  J. Disterhoft,et al.  Hippocampectomy disrupts trace eye-blink conditioning in rabbits. , 1990, Behavioral neuroscience.

[25]  J. Burgos Theoretical note: simulating latent inhibition with selection neural networks , 2003, Behavioural Processes.

[26]  W. Levy,et al.  A model of hippocampal activity in trace conditioning: where's the trace? , 2001, Behavioral neuroscience.

[27]  Guang-yan Wu,et al.  Reevaluating the Role of the Hippocampus in Delay Eyeblink Conditioning , 2013, PloS one.

[28]  L. Stein,et al.  Cellular investigations of behavioral reinforcement , 1989, Neuroscience & Biobehavioral Reviews.

[29]  Tadashi Yamazaki,et al.  A Neural Network Model for Trace Conditioning , 2004, ICONIP.

[30]  Juan M. Galeazzi,et al.  Some structural determinants of Pavlovian conditioning in artificial neural networks , 2010, Behavioural Processes.

[31]  S. Grossberg A neural model of attention, reinforcement and discrimination learning. , 1975, International review of neurobiology.

[32]  N. Mackintosh,et al.  Successful overshadowing and blocking in hippocampectomized rats , 1984, Behavioural Brain Research.

[33]  P. Solomon Role of the hippocampus in blocking and conditioned inhibition of the rabbit's nictitating membrane response. , 1977, Journal of comparative and physiological psychology.

[34]  R. Clark,et al.  Classical conditioning and brain systems: the role of awareness. , 1998, Science.

[35]  Tadashi Yamazaki,et al.  A Neural Network Model for Trace Conditioning , 2005, ICONIP.

[36]  Wulfram Gerstner,et al.  Theory and Simulation in Neuroscience , 2012, Science.

[37]  M. Spetch,et al.  Backward conditioning: a reevaluation of the empirical evidence. , 1981, Psychological bulletin.

[38]  J. Burgos Computational Models of Conditioning: The operant/respondent distinction: a computational neural-network analysis , 2010 .

[39]  J. Donahoe,et al.  Pavlovian conditioning: Pigeon nictitating membrane , 2011, Behavioural Processes.

[40]  E. Macphail,et al.  Hippocampal Lesions in Pigeons (Columba Livia) Disrupt Reinforced Preexposure but not Overshadowing or Blocking , 1994, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[41]  J. Donahoe,et al.  Unified principle of reinforcement in a neural-network model: Reply to N. T. Calvin and J. J. McDowell , 2016, Behavioural Processes.

[42]  M. Gluck,et al.  Why trace and delay conditioning are sometimes (but not always) hippocampal dependent: A computational model , 2013, Brain Research.

[43]  L. Swanson,et al.  Anatomical evidence for direct projections from the entorhinal area to the entire cortical mantle in the rat , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[44]  Louis D. Matzel,et al.  The Role of the Hippocampus in Trace Conditioning: Temporal Discontinuity or Task Difficulty? , 2001, Neurobiology of Learning and Memory.

[45]  J. Burgos,et al.  Autoshaped choice in artificial neural networks: Implications for behavioral economics and neuroeconomics , 2015, Behavioural Processes.

[46]  B. Berger,et al.  Morphological evidence for a dopaminergic terminal field in the hippocampal formation of young and adult rat , 1985, Neuroscience.

[47]  Richard F. Thompson,et al.  Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. , 1986, Behavioral neuroscience.

[48]  N. Schmajuk,et al.  Stimulus configuration, classical conditioning, and hippocampal function. , 1992, Psychological review.

[49]  W. B. Orr,et al.  Hippocampectomy selectively disrupts discrimination reversal conditioning of the rabbit nictitating membrane response , 1983, Behavioural Brain Research.

[50]  J. Burgos,et al.  A simultaneous procedure facilitates acquisition under an optimal interstimulus interval in artificial neural networks and rats , 2008, Behavioural Processes.

[51]  E. Coutureau,et al.  Entorhinal but Not Hippocampal or Subicular Lesions Disrupt Latent Inhibition in Rats , 1999, Neurobiology of Learning and Memory.

[52]  D. Alkon,et al.  Hippocampal lesions impair memory of short-delay conditioned eye blink in rabbits. , 1989, Behavioral neuroscience.

[53]  J. Disterhoft,et al.  Single Neurons in CA1 Hippocampus Encode Trace Interval Duration during Trace Heart Rate (Fear) Conditioning in Rabbit , 2003, The Journal of Neuroscience.

[54]  Mario Treviño,et al.  Controlled variations in stimulus similarity during learning determine visual discrimination capacity in freely moving mice , 2013, Scientific Reports.

[55]  Comparing excitatory backward and forward conditioning. , 2004, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[56]  R G M Morris,et al.  Elements of a neurobiological theory of the hippocampus: the role of activity-dependent synaptic plasticity in memory. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[57]  J. Burgos Autoshaping and automaintenance: a neural-network approach. , 2007, Journal of the experimental analysis of behavior.

[58]  J. Burgos,et al.  Neural-network simulations of two context-dependence phenomena , 2007, Behavioural Processes.

[59]  Mark G. Packard,et al.  Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat , 1994, Brain Research Bulletin.

[60]  N. Schmajuk,et al.  Absence of overshadowing in rats with hippocampal lesions , 1983 .

[61]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[62]  T. Shors,et al.  Trace Conditioning and the Hippocampus: The Importance of Contiguity , 2006, The Journal of Neuroscience.

[63]  Michael S Fanselow,et al.  Dorsal hippocampus involvement in trace fear conditioning with long, but not short, trace intervals in mice. , 2005, Behavioral neuroscience.

[64]  J. Burgos Chapter 4 – Evolving Artificial Neural Networks in Pavlovian Environments , 1997 .

[65]  J. Burgos Theoretical note: the C/T ratio in artificial neural networks , 2005, Behavioural Processes.

[66]  N. Schmajuk,et al.  A neural network approach to hippocampal function in classical conditioning. , 1991, Behavioral neuroscience.

[67]  W. Levy,et al.  A model of hippocampal activity in trace conditioning: where's the trace? , 2001 .

[68]  R. Insausti,et al.  Convergence of unimodal and polymodal sensory input to the entorhinal cortex in the fascicularis monkey , 2008, Neuroscience.

[69]  John F. Disterhoft,et al.  Increased Excitability of Aged Rabbit CA1 Neurons after Trace Eyeblink Conditioning , 2000, The Journal of Neuroscience.

[70]  H. Barbas,et al.  Topographically specific hippocampal projections target functionally distinct prefrontal areas in the rhesus monkey , 1995, Hippocampus.

[71]  Heith Cd,et al.  Simultaneous and backward fear conditioning in the rat. , 1973 .

[72]  Nicholas T. Calvin,et al.  Unified-theory-of-reinforcement neural networks do not simulate the blocking effect , 2015, Behavioural Processes.

[73]  W. K. Cullen,et al.  Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty , 2003, Nature Neuroscience.

[74]  Stephen Maren,et al.  Electrolytic lesions of the dorsal hippocampus disrupt renewal of conditional fear after extinction. , 2005, Learning & memory.

[75]  Richard S. Sutton,et al.  A computational model of hippocampal function in trace conditioning , 2008, NIPS.