Context Learning in the Rodent Hippocampus

We present a Bayesian statistical theory of context learning in the rodent hippocampus. While context is often defined in an experimental setting in relation to specific background cues or task demands, we advance a single, more general notion of context that suffices for a variety of learning phenomena. Specifically, a context is defined as a statistically stationary distribution of experiences, and context learning is defined as the problem of how to form contexts out of groups of experiences that cluster together in time. The challenge of context learning is solving the model selection problem: How many contexts make up the rodent's world? Solving this problem requires balancing two opposing goals: minimize the variability of the distribution of experiences within a context and minimize the likelihood of transitioning between contexts. The theory provides an understanding of why hippocampal place cell remapping sometimes develops gradually over many days of experience and why even consistent landmark differences may need to be relearned after other environmental changes. The theory provides an explanation for progressive performance improvements in serial reversal learning, based on a clear dissociation between the incremental process of context learning and the relatively abrupt context selection process. The impact of partial reinforcement on reversal learning is also addressed. Finally, the theory explains why alternating sequence learning does not consistently result in unique context-dependent sequence representations in hippocampus.

[1]  D W TYLER,et al.  Reversal-learning following partial reinforcement. , 1958, The American journal of psychology.

[2]  A. Lörincz,et al.  Two‐Phase Computational Model Training Long‐Term Memories in the Entorhinal‐Hippocampal Region , 2000, Annals of the New York Academy of Sciences.

[3]  Dietmar Schmitz,et al.  Presynaptic kainate receptors impart an associative property to hippocampal mossy fiber long-term potentiation , 2003, Nature Neuroscience.

[4]  M. Gluck,et al.  Dissociation of hippocampal and entorhinal function in associative learning: A computational approach , 1995, Psychobiology.

[5]  L. Jarrard On the role of the hippocampus in learning and memory in the rat. , 1993, Behavioral and neural biology.

[6]  B. H. Pubols,et al.  Serial reversal learning as a function of the number of trials per reversal. , 1962, Journal of comparative and physiological psychology.

[7]  G. Xavier,et al.  Dentate gyrus‐selective colchicine lesion and disruption of performance in spatial tasks: Difficulties in “place strategy” because of a lack of flexibility in the use of environmental cues? , 1999, Hippocampus.

[8]  H. Mallot,et al.  Learned association of allocentric and egocentric information in the hippocampus , 2004, Experimental Brain Research.

[9]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[10]  A. David Redish,et al.  The hippocampal debate: are we asking the right questions? , 2001, Behavioural Brain Research.

[11]  Bruce L. McNaughton,et al.  Path integration and the neural basis of the 'cognitive map' , 2006, Nature Reviews Neuroscience.

[12]  Kj Jeffery,et al.  Plasticity of the hippocampal cellular representation of place , 2000 .

[13]  M. Hasselmo,et al.  GABAergic modulation of hippocampal population activity: sequence learning, place field development, and the phase precession effect. , 1997, Journal of neurophysiology.

[14]  David S. Touretzky,et al.  Similarity and Discrimination in Classical Conditioning: A Latent Variable Account , 2004, NIPS.

[15]  I Neath,et al.  Remembering and forgetting as context discrimination. , 1995, Learning & memory.

[16]  R. Hirsh The hippocampus and contextual retrieval of information from memory: a theory. , 1974, Behavioral biology.

[17]  M. Hasselmo,et al.  The hippocampus as an associator of discontiguous events , 1998, Trends in Neurosciences.

[18]  L. Nadel,et al.  Context and conditioning: A place for space , 1980 .

[19]  R. J. McDonald,et al.  Attenuation of context-specific inhibition on reversal learning of a stimulus–response task in rats with neurotoxic hippocampal damage , 2002, Behavioural Brain Research.

[20]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[21]  J. Lisman,et al.  Dopamine Selectively Inhibits the Direct Cortical Pathway to the CA1 Hippocampal Region , 1999, The Journal of Neuroscience.

[22]  S. Wiener,et al.  Reward value invariant place responses and reward site associated activity in hippocampal neurons of behaving rats , 2003, Hippocampus.

[23]  Paul E. Gilbert,et al.  Dissociating hippocampal subregions: A double dissociation between dentate gyrus and CA1 , 2001, Hippocampus.

[24]  H. Eichenbaum,et al.  Hippocampal Neurons Encode Information about Different Types of Memory Episodes Occurring in the Same Location , 2000, Neuron.

[25]  Inah Lee,et al.  Differential roles of dorsal hippocampal subregions in spatial working memory with short versus intermediate delay. , 2003, Behavioral neuroscience.

[26]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[27]  J. Knierim,et al.  Comparison of population coherence of place cells in hippocampal subfields CA1 and CA3 , 2004, Nature.

[28]  C Kentros,et al.  Abolition of long-term stability of new hippocampal place cell maps by NMDA receptor blockade. , 1998, Science.

[29]  M Tsodyks,et al.  Attractor neural network models of spatial maps in hippocampus , 1999, Hippocampus.

[30]  M. Shapiro,et al.  Prospective and Retrospective Memory Coding in the Hippocampus , 2003, Neuron.

[31]  H. Eichenbaum,et al.  Cues that hippocampal place cells encode: Dynamic and hierarchical representation of local and distal stimuli , 1997, Hippocampus.

[32]  W E Skaggs,et al.  Interactions between location and task affect the spatial and directional firing of hippocampal neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  E. Brunswik Probability as a determiner of rat behavior. , 1939 .

[34]  Marc W Howard,et al.  The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains. , 2005, Psychological review.

[35]  A. Redish Beyond the Cognitive Map: From Place Cells to Episodic Memory , 1999 .

[36]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[37]  M. Gluck,et al.  Hippocampal mediation of stimulus representation: A computational theory , 1993, Hippocampus.

[38]  Joe Z Tsien,et al.  Deficient Neurogenesis in Forebrain-Specific Presenilin-1 Knockout Mice Is Associated with Reduced Clearance of Hippocampal Memory Traces , 2001, Neuron.

[39]  E. Bostock,et al.  Experience‐dependent modifications of hippocampal place cell firing , 1991, Hippocampus.

[40]  D. Kimble,et al.  Brightness discrimination and reversal in hippocampally-lesioned rats , 1968 .

[41]  N. Schmajuk,et al.  Stimulus configuration, classical conditioning, and hippocampal function. , 1992, Psychological review.

[42]  Paul E. Gilbert,et al.  The role of the hippocampus in memory for the temporal order of a sequence of odors. , 2002, Behavioral neuroscience.

[43]  Joel E. Brown,et al.  Concordant and discordant coding of spatial location in populations of hippocampal CA1 pyramidal cells. , 2002, Journal of neurophysiology.

[44]  W. B. Orr,et al.  Hippocampectomy selectively disrupts discrimination reversal conditioning of the rabbit nictitating membrane response , 1983, Behavioural Brain Research.

[45]  W. Abraham,et al.  Induction and Experience-Dependent Consolidation of Stable Long-Term Potentiation Lasting Months in the Hippocampus , 2002, The Journal of Neuroscience.

[46]  E. Rolls,et al.  Computational analysis of the role of the hippocampus in memory , 1994, Hippocampus.

[47]  L M WISE Supplementary report: the Weinstock partial reinforcement effect and habit reversal. , 1962, Journal of experimental psychology.

[48]  David S. Touretzky,et al.  Model Uncertainty in Classical Conditioning , 2003, NIPS.

[49]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[50]  C. Gallistel,et al.  Time, rate, and conditioning. , 2000, Psychological review.

[51]  J. M. Novak,et al.  Serial position curve in rats: role of the dorsal hippocampus. , 1982, Science.

[52]  John M. Olin Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .

[53]  Thomas J. Wills,et al.  Long-term plasticity in hippocampal place-cell representation of environmental geometry , 2002, Nature.

[54]  M. Hasselmo,et al.  Dynamics of learning and recall at excitatory recurrent synapses and cholinergic modulation in rat hippocampal region CA3 , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[55]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[56]  E. L. Wike,et al.  Extinction of a partially and continuously reinforced response with and without a rewarded alternative. , 1953, Journal of experimental psychology.

[57]  G. Kimble,et al.  One-trial discrimination reversal in the white rat. , 1954, Journal of comparative and physiological psychology.

[58]  H. Eichenbaum,et al.  The global record of memory in hippocampal neuronal activity , 1999, Nature.

[59]  H. Eichenbaum,et al.  Discordance of spatial representation in ensembles of hippocampal place cells , 1997, Hippocampus.

[60]  K. Jeffery,et al.  Context‐specific acquisition of location discrimination by hippocampal place cells , 2003, The European journal of neuroscience.

[61]  E. Rolls A theory of hippocampal function in memory , 1996, Hippocampus.

[62]  E. Save,et al.  Place‐cell firing does not depend on the direction of turn in a Y‐maze alternation task , 2001, The European journal of neuroscience.

[63]  H. Eichenbaum,et al.  Critical role of the hippocampus in memory for sequences of events , 2002, Nature Neuroscience.

[64]  S. Frühwirth-Schnatter Estimating Marginal Likelihoods for Mixture and Markov Switching Models Using Bridge Sampling Techniques , 2004 .

[65]  E. Capaldi The sequential view: From rapidly fading stimulus traces to the organization of memory and the abstract concept of number , 1994, Psychonomic bulletin & review.

[66]  David S. Touretzky,et al.  The Role of the Hippocampus in Solving the Morris Water Maze , 1998, Neural Computation.

[67]  J. Lisman,et al.  Inhibition of perforant path input to the CA1 region by serotonin and noradrenaline. , 2005, Journal of neurophysiology.

[68]  Daniel L. Schacter,et al.  Spatial Representation in the Entorhinal Cortex , 2004 .

[69]  H. Eichenbaum,et al.  The Hippocampus and Disambiguation of Overlapping Sequences , 2002, The Journal of Neuroscience.

[70]  Mark C. Fuhs,et al.  A Spin Glass Model of Path Integration in Rat Medial Entorhinal Cortex , 2006, The Journal of Neuroscience.

[71]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[72]  M. Hasselmo,et al.  Laminar selectivity of the cholinergic suppression of synaptic transmission in rat hippocampal region CA1: computational modeling and brain slice physiology , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[73]  B. McNaughton,et al.  Multistability of cognitive maps in the hippocampus of old rats , 1997, Nature.

[74]  D. Kimble,et al.  Hippocampectomy and response perseveration in the rat. , 1965, Journal of comparative and physiological psychology.

[75]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[76]  Michael E. Hasselmo,et al.  A Proposed Function for Hippocampal Theta Rhythm: Separate Phases of Encoding and Retrieval Enhance Reversal of Prior Learning , 2002, Neural Computation.

[77]  R. Muller,et al.  The positional firing properties of medial entorhinal neurons: description and comparison with hippocampal place cells , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[78]  M. Hasselmo,et al.  Encoding and retrieval of episodic memories: Role of cholinergic and GABAergic modulation in the hippocampus , 1998, Hippocampus.

[79]  J. Ison,et al.  Effects of partial reinforcement on discrimination learning and subsequent reversal or extinction. , 1967, Journal of experimental psychology.

[80]  David M. Smith,et al.  Learning-Related Development of Context-Specific Neuronal Responses to Places and Events: The Hippocampal Role in Context Processing , 2006, The Journal of Neuroscience.

[81]  R P Kesner,et al.  Memory for spatial location as a function of temporal lag in rats: role of hippocampus and medial prefrontal cortex. , 1994, Behavioral and neural biology.

[82]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[83]  Aaron C. Courville,et al.  A latent cause theory of classical conditioning , 2006 .

[84]  B. McNaughton,et al.  Hippocampal synaptic enhancement and information storage within a distributed memory system , 1987, Trends in Neurosciences.

[85]  R. Sutherland,et al.  Configural association theory and the hippocampal formation: An appraisal and reconfiguration , 1995, Hippocampus.

[86]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[87]  W B Levy,et al.  A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal‐like tasks , 1996, Hippocampus.

[88]  E. Save,et al.  Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[89]  E T Rolls,et al.  Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network , 1992, Hippocampus.

[90]  M. Quirk,et al.  Requirement for Hippocampal CA3 NMDA Receptors in Associative Memory Recall , 2002, Science.

[91]  A. Treves,et al.  Distinct Ensemble Codes in Hippocampal Areas CA3 and CA1 , 2004, Science.

[92]  M. Gluck,et al.  Integrating behavioral and physiological models of hippocampal function , 1996, Hippocampus.

[93]  B. McNaughton,et al.  Spatial Firing Properties of Hippocampal CA1 Populations in an Environment Containing Two Visually Identical Regions , 1998, The Journal of Neuroscience.

[94]  Michael E. Hasselmo,et al.  Hippocampal Mechanisms for the Context-dependent Retrieval of Episodes Action Reward Place (state) , 2022 .

[95]  James L. McClelland,et al.  Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade‐off , 1994, Hippocampus.

[96]  D S Touretzky,et al.  Theory of rodent navigation based on interacting representations of space , 1996, Hippocampus.

[97]  Ali A. Minai,et al.  Latent Attractors: A Model for Context-Dependent Place Representations in the Hippocampus , 2000, Neural Computation.

[98]  W. Levy A computational approach to hippocampal function , 1989 .

[99]  Inah Lee,et al.  Differential contribution of NMDA receptors in hippocampal subregions to spatial working memory , 2002, Nature Neuroscience.

[100]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[101]  Bruce L. McNaughton,et al.  Progressive Transformation of Hippocampal Neuronal Representations in “Morphed” Environments , 2005, Neuron.

[102]  J. Aggleton,et al.  Lack of effect of lesions in the anterior cingulate cortex and retrosplenial cortex on certain tests of spatial memory in the rat , 1994, Behavioural Brain Research.

[103]  R. Muller,et al.  The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[104]  N. Schmajuk,et al.  A neural network approach to hippocampal function in classical conditioning. , 1991, Behavioral neuroscience.

[105]  Gordon H. Bower,et al.  Computational models of learning in simple neural systems , 1989 .

[106]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[107]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[108]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[109]  James L Olds,et al.  Effects of context manipulation on memory and reversal learning in rats with hippocampal lesions. , 1978, Journal of comparative and physiological psychology.

[110]  David S. Touretzky,et al.  Representation and Timing in Theories of the Dopamine System , 2006, Neural Computation.

[111]  B. McNaughton,et al.  Place cells, head direction cells, and the learning of landmark stability , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[112]  M. Wilson,et al.  Trajectory Encoding in the Hippocampus and Entorhinal Cortex , 2000, Neuron.

[113]  David S. Touretzky,et al.  Synaptic learning models of map separation in the hippocampus , 2000, Neurocomputing.

[114]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[115]  James J. Knierim,et al.  Ensemble Dynamics of Hippocampal Regions CA3 and CA1 , 2004, Neuron.

[116]  J. Guzowski,et al.  Differences in Hippocampal Neuronal Population Responses to Modifications of an Environmental Context: Evidence for Distinct, Yet Complementary, Functions of CA3 and CA1 Ensembles , 2004, The Journal of Neuroscience.

[117]  Mark R. Bower,et al.  Sequential-Context-Dependent Hippocampal Activity Is Not Necessary to Learn Sequences with Repeated Elements , 2005, The Journal of Neuroscience.

[118]  M. Andersson,et al.  Independent Codes for Spatial and Episodic Memory in Hippocampal Neuronal Ensembles , 2005 .

[119]  J. Hall,et al.  Reinforcement schedules in habit reversal; a confirmation. , 1954, Journal of experimental psychology.