Dissociable effects of surprising rewards on learning and memory

Reward-prediction errors track the extent to which rewards deviate from expectations, and aid in learning. How do such errors in prediction interact with memory for the rewarding episode? Existing findings point to both cooperative and competitive interactions between learning and memory mechanisms. Here, we investigated whether learning about rewards in a high-risk context, with frequent, large prediction errors, would give rise to higher fidelity memory traces for rewarding events than learning in a low-risk context. Experiment 1 showed that recognition was better for items associated with larger absolute prediction errors during reward learning. Larger prediction errors also led to higher rates of learning about rewards. Interestingly we did not find a relationship between learning rate for reward and recognition-memory accuracy for items, suggesting that these two effects of prediction errors were caused by separate underlying mechanisms. In Experiment 2, we replicated these results with a longer task that posed stronger memory demands and allowed for more learning. We also showed improved source and sequence memory for items within the high-risk context. In Experiment 3, we controlled for the difficulty of reward learning in the risk environments, again replicating the previous results. Moreover, this control revealed that the high-risk context enhanced item-recognition memory beyond the effect of prediction errors. In summary, our results show that prediction errors boost both episodic item memory and incremental reward learning, but the two effects are likely mediated by distinct underlying systems.

[1]  D. Bavelier,et al.  Influence of reward motivation on human declarative memory , 2016, Neuroscience & Biobehavioral Reviews.

[2]  W. Schultz,et al.  Adaptive Prediction Error Coding in the Human Midbrain and Striatum Facilitates Behavioral Adaptation and Learning Efficiency , 2016, Neuron.

[3]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[4]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[5]  L. Davachi,et al.  The influence of context boundaries on memory for the sequential order of events. , 2013, Journal of experimental psychology. General.

[6]  Erin Kendall Braun,et al.  A Trade-Off between Feedback-Based Learning and Episodic Memory for Feedback Events: Evidence from Parkinson’s Disease , 2012, Neurodegenerative Diseases.

[7]  Panagiotis G. Ipeirotis Analyzing the Amazon Mechanical Turk marketplace , 2010, XRDS.

[8]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[9]  Jennifer A. Mangels,et al.  Neural correlates of error detection and correction in a semantic retrieval task. , 2003, Brain research. Cognitive brain research.

[10]  Donald A. Norman,et al.  A non-parametric analysis of recognition experiments , 1964 .

[11]  Wolfram Schultz,et al.  Scaling prediction errors to reward variability benefits error-driven learning in humans , 2015, Journal of neurophysiology.

[12]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[13]  Alison Adcock,et al.  Enriched encoding: reward motivation organizes cortical networks for hippocampal detection of unexpected events. , 2014, Cerebral cortex.

[14]  S. Sara The locus coeruleus and noradrenergic modulation of cognition , 2009, Nature Reviews Neuroscience.

[15]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[16]  Lisa K. Fazio,et al.  Correcting False Memories , 2010, Psychological science.

[17]  Erin Kendall Braun,et al.  Episodic Memory Encoding Interferes with Reward Learning and Decreases Striatal Prediction Errors , 2014, The Journal of Neuroscience.

[18]  M. Mather,et al.  Locus coeruleus neuromodulation of memories encoded during negative or unexpected action outcomes , 2014, Neurobiology of Learning and Memory.

[19]  Eduardo E. Benarroch,et al.  Locus coeruleus , 2017, Cell and Tissue Research.

[20]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[21]  E. Kandel,et al.  Dopamine release from the locus coeruleus to the dorsal hippocampus promotes spatial learning and memory , 2016, Proceedings of the National Academy of Sciences.

[22]  Mara Mather,et al.  Cognitive control, dynamic salience, and the imperative toward computational accounts of neuromodulatory function , 2015, Behavioral and Brain Sciences.

[23]  G. Nahler correction of errors , 2009 .

[24]  Marcia L. Spetch,et al.  Remembering the best and worst of times: Memories for extreme outcomes bias risky decisions , 2013, Psychonomic Bulletin & Review.

[25]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[26]  Lindsay E. Hunter,et al.  Episodic memories predict adaptive value-based decision-making. , 2016, Journal of experimental psychology. General.

[27]  J. Metcalfe,et al.  The correction of errors committed with high confidence , 2006 .

[28]  Marcia L. Spetch,et al.  Extreme outcomes sway risky decisions from experience , 2014 .

[29]  R. Henson,et al.  Does prediction error drive one-shot declarative learning? , 2017, Journal of memory and language.

[30]  R. K. Simpson Nature Neuroscience , 2022 .

[31]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[32]  Brian Knutson,et al.  Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation , 2006, Neuron.

[33]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[34]  K. Duncan,et al.  Memory’s Penumbra: Episodic Memory Decisions Induce Lingering Mnemonic Biases , 2012, Science.

[35]  E. Weber,et al.  A Domain-Specific Risk-Attitude Scale: Measuring Risk Perceptions and Risk Behaviors , 2002 .

[36]  Alexander Strashny Asymmetric loss utility: an analysis of decision under risk , 2004 .

[37]  Colin Camerer,et al.  Neural Response to Reward Anticipation under Risk Is Nonlinear in Probabilities , 2009, The Journal of Neuroscience.

[38]  Lisa K. Fazio,et al.  Surprising feedback improves later memory , 2009, Psychonomic bulletin & review.

[39]  J. Metcalfe,et al.  Errors committed with high confidence are hypercorrected. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[40]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[41]  R. Morris,et al.  Locus coeruleus and dopaminergic consolidation of everyday memory , 2016, Nature.

[42]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[43]  K Richard Ridderinkhof,et al.  Adaptive Coding , 2005, Science.

[44]  D. Shohamy,et al.  Memory states influence value-based decisions. , 2016, Journal of experimental psychology. General.

[45]  Samuel J. Gershman,et al.  Perceptual estimation obeys Occam's razor , 2013, Front. Psychol..

[46]  P. Dayan,et al.  Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain , 2012, The Journal of Neuroscience.

[47]  Panagiotis G. Ipeirotis,et al.  The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk , 2015, WWW.

[48]  I. Erev,et al.  Learning, risk attitude and hot stoves in restless bandit problems , 2009 .