A Normative Theory of Forgetting: Lessons from the Fruit Fly

Recent experiments revealed that the fruit fly Drosophila melanogaster has a dedicated mechanism for forgetting: blocking the G-protein Rac leads to slower and activating Rac to faster forgetting. This active form of forgetting lacks a satisfactory functional explanation. We investigated optimal decision making for an agent adapting to a stochastic environment where a stimulus may switch between being indicative of reward or punishment. Like Drosophila, an optimal agent shows forgetting with a rate that is linked to the time scale of changes in the environment. Moreover, to reduce the odds of missing future reward, an optimal agent may trade the risk of immediate pain for information gain and thus forget faster after aversive conditioning. A simple neuronal network reproduces these features. Our theory shows that forgetting in Drosophila appears as an optimal adaptive behavior in a changing environment. This is in line with the view that forgetting is adaptive rather than a consequence of limitations of the memory system.

[1]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[2]  J. Wixted CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE A Theory About Why We Forget What We Once Knew , 2022 .

[3]  Russ Kahan,et al.  Opportunity costs , 1998 .

[4]  Aaron C. Courville,et al.  Bayesian theories of conditioning in a changing world , 2006, Trends in Cognitive Sciences.

[5]  Gabriel Kreiman Mind the quantum? Werner R. Loewenstein Physics in Mind: A Quantum View of the Brain , 2013, Trends in Cognitive Sciences.

[6]  Binyan Lu,et al.  Forgetting Is Regulated through Rac Activity in Drosophila , 2010, Cell.

[7]  Ronald L. Davis,et al.  Distinct Traces for Appetitive versus Aversive Olfactory Memories in DPM Neurons of Drosophila , 2012, Current Biology.

[8]  K. M. Dallenbach,et al.  Obliviscence During Sleep and Waking. , 1924 .

[9]  K. Doya Reinforcement learning: Computational theory and biological mechanisms , 2007, HFSP journal.

[10]  G. Riano,et al.  Linear Programming solvers for Markov Decision Processes , 2006, 2006 IEEE Systems and Information Engineering Design Symposium.

[11]  Michael L. Littman,et al.  Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes , 1997, UAI.

[12]  C. D. Beck,et al.  Learning Performance of Normal and MutantDrosophila after Repeated Conditioning Trials with Discrete Stimuli , 2000, The Journal of Neuroscience.

[13]  Jonathan M. Golding,et al.  Adaptive forgetting in animals , 1997 .

[14]  Benjamin C. Storm The Benefit of Forgetting in Thinking and Remembering , 2011 .

[15]  C. Gallistel,et al.  The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. , 2001, Journal of experimental psychology. Animal behavior processes.

[16]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[17]  W. Quinn,et al.  Classical conditioning and retention in normal and mutantDrosophila melanogaster , 1985, Journal of Comparative Physiology A.

[18]  K. Arrow,et al.  The New Palgrave Dictionary of Economics , 2020 .

[19]  Ronald L. Davis,et al.  Dopamine Is Required for Learning and Forgetting in Drosophila , 2012, Neuron.

[20]  R. Menzel,et al.  Massed and spaced learning in honeybees: the role of CS, US, the intertrial interval, and the test interval. , 2001, Learning & memory.

[21]  T. Préat,et al.  Genetic dissection of consolidated memory in Drosophila , 1994, Cell.

[22]  W. Quinn,et al.  Reward learning in normal and mutant Drosophila. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[23]  J. Wixted The psychology and neuroscience of forgetting. , 2004, Annual review of psychology.

[24]  Charles H. Judd,et al.  The Fundamentals of Learning. Edward L. Thorndike , The Staff of the Division of Psychology of the Institute of Educational Research of Teachers College, Columbia UniversityHuman Learning. Edward L. Thorndike , 1933 .

[25]  James Flynn Averaging vs. Discounting in Dynamic Programming: a Counterexample , 1974 .

[26]  Peter Bossaerts,et al.  Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings , 2011, PLoS Comput. Biol..

[27]  Robert C. Wilson,et al.  An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing Environment , 2010, The Journal of Neuroscience.

[28]  M E Bitterman,et al.  Reversal Learning and Forgetting in Bird and Fish , 1967, Science.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  Joseph D Hagman,et al.  Effects of Training Schedule and Equipment Variety on Retention and Transfer of Maintenance Skill , 1980 .

[31]  L. Nadel,et al.  Decay happens: the role of active forgetting in memory , 2013, Trends in Cognitive Sciences.