A Computational Model for Latent Learning based on Hippocampal Replay

We show how hippocampal replay could explain latent learning, a phenomenon observed in animals where unrewarded pre-exposure to an environment, i.e. habituation, improves task learning rates once rewarded trials begin. We first describe a computational model for spatial navigation inspired by rat studies. The model exploits offline replay of trajectories previously learned by applying reinforcement learning. Then, to assess our hypothesis, the model is evaluated in a "multiple T-maze" environment where rats need to learn a path from the start of the maze to the goal. Simulation results support our hypothesis that pre-exposed or habituated rats learn the task significantly faster than non-pre-exposed rats. Results also show that this effect increases with the number of pre-exposed trials.

[1]  RU Muller,et al.  The hippocampus as a cognitive graph , 1996, The Journal of general physiology.

[2]  M. Botvinick,et al.  The hippocampus as a predictive map , 2016 .

[3]  D. Olton.  Mazes, maps, and memory. , 1979, The American psychologist.

[4]  J. Born,et al.  Neuroscience and Biobehavioral Reviews Sleep and Memory in Mammals, Birds and Invertebrates , 2022 .

[5]  O. J. Dunn.  Multiple Comparisons Using Rank Sums , 1964 .

[6]  L. F. Abbott,et al.  A Model of Spatial Map Formation in the Hippocampus of the Rat , 1999, Neural Computation.

[7]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[10]  W. S. Small.  Experimental Study of the Mental Processes of the Rat. II , 1901 .

[11]  Margaret F. Carr,et al.  Hippocampal SWR Activity Predicts Correct Decisions during the Initial Learning of an Alternation Task , 2013, Neuron.

[12]  A. David Redish,et al.  Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model , 2005, Neural Networks.

[13]  Y. Benjamini.  Discovering the false discovery rate , 2010 .

[14]  E. Thorndike.  Experimental Study of the Mental Processes of the Rat. , 1901 .

[15]  A. David Redish,et al.  The role of the hippocampus in the Morris water maze , 1998 .

[16]  Margaret F. Carr,et al.  Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval , 2011, Nature Neuroscience.

[17]  J. I.  The Design of Experiments , 1936, Nature.

[18]  B. McNaughton,et al.  Memory trace reactivation in hippocampal and neocortical neuronal ensembles , 2000, Current Opinion in Neurobiology.

[19]  B. McNaughton,et al.  Reactivation of hippocampal ensemble memories during sleep. , 1994, Science.

[20]  H. Blodgett,et al.  The effect of the introduction of reward upon the maze performance of rats , 1929 .

[21]  Peter Dayan,et al.  Improving Generalization for Temporal Difference Learning: The Successor Representation , 1993, Neural Computation.

[22]  Michael A. Arbib,et al.  Affordances. Motivations, and the World Graph Theory , 1998, Adapt. Behav..

[23]  K. Spence,et al.  An experimental test of the sign-gestalt theory of trial and error learning. , 1946 .

[24]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[25]  B. McNaughton,et al.  Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences , 1996, Hippocampus.