Memory Alone Does Not Account for the Way Rats Learn a Simple Spatial Alternation Task

Animal behavior provides context for understanding disease models and physiology. However, that behavior is often characterized subjectively, creating opportunity for misinterpretation and misunderstanding. For example, spatial alternation tasks are treated as paradigmatic tools for examining memory; however, that link is actually an assumption. To test this assumption, we simulated a reinforcement learning (RL) agent equipped with a perfect memory process. We found that it learns a simple spatial alternation task more slowly and makes different errors than a group of male rats, illustrating that memory alone may not be sufficient to capture the behavior. We demonstrate that incorporating spatial biases permits rapid learning and enables the model to fit rodent behavior accurately. Our results suggest that even simple spatial alternation behaviors reflect multiple cognitive processes that need to be taken into account when studying animal behavior. SIGNIFICANCE STATEMENT Memory is a critical function for cognition whose impairment has significant clinical consequences. Experimental systems aimed at testing various sorts of memory are therefore also central. However, experimental designs to test memory are typically based on intuition about the underlying processes. We tested this using a popular behavioral paradigm: a spatial alternation task. Using behavioral modeling, we show that the straightforward intuition that these tasks just probe spatial memory fails to account for the speed at which rats learn or the types of errors they make. Only when memory-independent dynamic spatial preferences are added can the model learn like the rats. This highlights the importance of respecting the complexity of animal behavior to interpret neural function and validate disease models.

[1]  R. Porsolt,et al.  Behavioural despair in rats: a new model sensitive to antidepressant treatments. , 1978, European journal of pharmacology.

[2]  R. Morris Developments of a water-maze procedure for studying spatial learning in the rat , 1984, Journal of Neuroscience Methods.

[3]  S. File,et al.  Validation of open : closed arm entries in an elevated plus-maze as a measure of anxiety in the rat , 1985, Journal of Neuroscience Methods.

[4]  J. Aggleton,et al.  A comparison of the effects of anterior thalamic, mamillary body and fornix lesions on reinforced spatial alternation , 1995, Behavioural Brain Research.

[5]  P. Gisquet-Verrier,et al.  Prelimbic cortex specific lesions disrupt delayed-variable response tasks in the rat. , 1996, Behavioral neuroscience.

[6]  J. Aggleton,et al.  Spontaneous object recognition and object location memory in rats: the effects of lesions in the cingulate cortices, the medial prefrontal cortex, the cingulum bundle and the fornix , 1997, Experimental Brain Research.

[7]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[8]  M. Wilson,et al.  Trajectory Encoding in the Hippocampus and Entorhinal Cortex , 2000, Neuron.

[9]  N. Swerdlow,et al.  Animal models of deficient sensorimotor gating: what we know, what we think we know, and what we hope to know soon. , 2000, Behavioural pharmacology.

[10]  P. Gisquet-Verrier,et al.  Involvement of the dorsal anterior cingulate cortex in temporal behavioral sequencing: subregional analysis of the medial prefrontal cortex in rat , 2001, Behavioural Brain Research.

[11]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[12]  H. Mallot,et al.  Inactivation of the rat dorsal striatum impairs performance in spatial tasks and alters hippocampal theta in the freely moving rat , 2005, Behavioural Brain Research.

[13]  P. Gisquet-Verrier,et al.  The role of the rat prelimbic/infralimbic cortex in working memory: Not involved in the short-term maintenance but in monitoring and processing functions , 2006, Neuroscience.

[14]  C. Vorhees,et al.  Morris water maze: procedures for assessing spatial and related forms of learning and memory , 2006, Nature Protocols.

[15]  A. Walf,et al.  The use of the elevated plus maze as an assay of anxiety-related behavior in rodents , 2007, Nature Protocols.

[16]  Mattias P. Karlsson,et al.  Network Dynamics Underlying the Formation of Sparse, Informative Representations in the Hippocampus , 2008, The Journal of Neuroscience.

[17]  Jonathan D. Cohen,et al.  Learning to Use Working Memory in Partially Observable Environments through Dopaminergic Reinforcement , 2008, NIPS.

[18]  Eric A. Zilli,et al.  Modeling the role of working memory and episodic memory in behavioral tasks , 2008, Hippocampus.

[19]  W. Gerstner,et al.  Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning , 2009, Nature Neuroscience.

[20]  L. Frank,et al.  Hippocampal Lesions Impair Rapid Learning of a Continuous Spatial Alternation Task , 2009, PloS one.

[21]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[22]  J. Gordon,et al.  Impaired hippocampal–prefrontal synchrony in a genetic mouse model of schizophrenia , 2010, Nature.

[23]  B. Poucet,et al.  Contributions of dorsal striatal subregions to spatial alternation behavior. , 2011, Learning & memory.

[24]  Nathaniel D. Daw,et al.  Trial-by-trial data analysis using computational models , 2011 .

[25]  T. Robbins,et al.  Decision Making, Affect, and Learning: Attention and Performance XXIII , 2011 .

[26]  Bridget Valsamis,et al.  Habituation and prepulse inhibition of acoustic startle in rodents. , 2011, Journal of visualized experiments : JoVE.

[27]  D. Slattery,et al.  Using the rat forced swim test to assess antidepressant-like activity in rodents , 2012, Nature Protocols.

[28]  Keizo Takao,et al.  T-maze Forced Alternation and Left-right Discrimination Tasks for Assessing Working and Reference Memory in Mice , 2012, Journal of visualized experiments : JoVE.

[29]  Margaret F. Carr,et al.  Transient Slow Gamma Synchrony Underlies Hippocampal Memory Replay , 2012, Neuron.

[30]  Rafal Bogacz,et al.  Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats , 2012, Front. Comput. Neurosci..

[31]  L. Frank,et al.  Awake Hippocampal Sharp-Wave Ripples Support Spatial Memory , 2012, Science.

[32]  Demetris K. Roumis,et al.  Coordinated Excitation and Inhibition of Prefrontal Ensembles during Awake Hippocampal Sharp-Wave Ripple Events , 2016, Neuron.

[33]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[34]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[35]  M. Gutmann,et al.  Fundamentals and Recent Developments in Approximate Bayesian Computation , 2016, Systematic biology.

[36]  Bin Xu,et al.  Recapitulation and Reversal of Schizophrenia-Related Phenotypes in Setd1a-Deficient Mice , 2019, Neuron.

[37]  Wenbo Tang,et al.  Dynamics of Awake Hippocampal-Prefrontal Replay for Spatial Learning and Memory-Guided Decision Making , 2019, Neuron.

[38]  G. Buzsáki,et al.  Long-duration hippocampal sharp wave ripples improve memory , 2019, Science.