A common mechanism for adaptive scaling of reward and novelty

Declarative memory is remarkably adaptive in the way it maintains sensitivity to relative novelty in both unknown and highly familiar environments. However, the neural mechanisms underlying this contextual adaptation are poorly understood. On the basis of emerging links between novelty processing and reinforcement learning mechanisms, we hypothesized that responses to novelty will be adaptively scaled according to expected contextual probabilities of new and familiar events, in the same way that responses to prediction errors for rewards are scaled according to their expected range. Using functional magnetic resonance imaging in humans, we show that the influence of novelty and reward on memory formation in an incidental memory task is adaptively scaled and furthermore that the BOLD signal in orbital prefrontal and medial temporal cortices exhibits concomitant scaled adaptive coding. These findings demonstrate a new mechanism for adjusting gain and sensitivity in declarative memory in accordance with contextual probabilities and expectancies of future events. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.

[1]  E. N. Sokolov Higher nervous functions; the orienting reflex. , 1963, Annual review of physiology.

[2]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[3]  L. Devenport,et al.  Reward-induced stereotypy: modulation by the hippocampus. , 1981, Science.

[4]  M. Geyer,et al.  Behavior during hippocampal microinfusions. II. Muscarinic locomotor activation , 1982, Brain Research Reviews.

[5]  E. Balint Memory and consciousness. , 1987, The International journal of psycho-analysis.

[6]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[7]  R. Desimone,et al.  Parallel neuronal mechanisms for short-term memory. , 1994, Science.

[8]  Mark G. Packard,et al.  Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat , 1994, Brain Research Bulletin.

[9]  E. Tulving,et al.  Novelty assessment in the brain and long-term memory encoding , 1995, Psychonomic bulletin & review.

[10]  G. Hall,et al.  Enhancement of latent inhibition in rats with electrolytic lesions of the hippocampus. , 1995, Behavioral neuroscience.

[11]  L. Jacoby,et al.  The Relation between Remembering and Knowing as Bases for Recognition: Effects of Size Congruency , 1995 .

[12]  R. Knight Contribution of human hippocampal region to novelty detection , 1996, Nature.

[13]  R. Desimone,et al.  Neural mechanisms for visual memory and their role in attention. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[14]  F. Craik,et al.  Novelty and familiarity activations in PET studies of memory encoding and retrieval. , 1996, Cerebral cortex.

[15]  J. Cohen,et al.  Schizophrenic deficits in the processing of context. A test of a theoretical model. , 1996, Archives of general psychiatry.

[16]  Mark G. Packard,et al.  The dopaminergic mesencephalic projections to the hippocampal formation in the rat , 1997, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[17]  M. Hasselmo,et al.  Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function , 1997, Behavioural Brain Research.

[18]  R. Turner,et al.  Event-Related fMRI: Characterizing Differential Responses , 1998, NeuroImage.

[19]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[20]  Karl J. Friston,et al.  Characterizing Stimulus–Response Functions Using Nonlinear Regressors in Parametric fMRI Experiments , 1998, NeuroImage.

[21]  M. W. Brown,et al.  Recognition memory: neuronal substrates of the judgement of prior occurrence , 1998, Progress in Neurobiology.

[22]  U. Frey,et al.  Synaptic tagging: implications for late maintenance of hippocampal long-term potentiation , 1998, Trends in Neurosciences.

[23]  D G Gadian,et al.  Amnesia and the organization of the hippocampal system , 1998, Hippocampus.

[24]  R. C. Honey,et al.  Hippocampal Lesions Disrupt an Associative Mismatch Process , 1998, The Journal of Neuroscience.

[25]  J. Ashburner,et al.  Nonlinear spatial normalization using basis functions , 1999, Human brain mapping.

[26]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[27]  P. Matthews,et al.  Learning about pain: the neural substrate of the prediction error for aversive events. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Y. Smith,et al.  Anatomy of the dopamine system in the basal ganglia , 2000, Trends in Neurosciences.

[29]  A. Dale,et al.  Deconvolution of Event-Related fMRI Responses in Fast-Rate Experimental Designs: Tracking Amplitude Variations , 2000, Journal of Cognitive Neuroscience.

[30]  S. Wiener,et al.  Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats , 2000, Hippocampus.

[31]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[32]  J. Lisman,et al.  Hippocampus as comparator: Role of the two input and two output systems of the hippocampus in selection and registration of information , 2001, Hippocampus.

[33]  Karl J. Friston,et al.  Modelling Geometric Deformations in Epi Time Series , 2022 .

[34]  Randy L. Buckner,et al.  Encoding Processes during Retrieval Tasks , 2001, Journal of Cognitive Neuroscience.

[35]  Malcolm W. Brown,et al.  Recognition memory: What are the roles of the perirhinal cortex and hippocampus? , 2001, Nature Reviews Neuroscience.

[36]  M Mishkin,et al.  Brain activity evidence for recognition without recollection after early hippocampal damage , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[37]  R. Turner,et al.  Combined correction for geometric distortion and its interaction with head movement in fMRI , 2002 .

[38]  Robert Turner,et al.  Image Distortion Correction in fMRI: A Quantitative Evaluation , 2002, NeuroImage.

[39]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[40]  Robert T. Knight,et al.  Effects of extensive temporal lobe damage or mild hypoxia on recollection and familiarity , 2002, Nature Neuroscience.

[41]  H. Heinze,et al.  The effect of item sequence on brain activity during recognition memory. , 2002, Brain research. Cognitive brain research.

[42]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[43]  Vincent Walsh A theory of magnitude: common cortical metrics of time, space and quantity , 2003, Trends in Cognitive Sciences.

[44]  R Turner,et al.  Optimized EPI for fMRI studies of the orbitofrontal cortex , 2003, NeuroImage.

[45]  H. Mallot,et al.  Reward modulates neuronal activity in the hippocampus of the rat , 2003, Behavioural Brain Research.

[46]  W. K. Cullen,et al.  Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty , 2003, Nature Neuroscience.

[47]  Hans-Jochen Heinze,et al.  Human Hippocampal and Parahippocampal Activity during Visual Associative Recognition Memory for Spatial and Nonspatial Stimulus Configurations , 2003, The Journal of Neuroscience.

[48]  G. Orban,et al.  Parietal Representation of Symbolic and Nonsymbolic Magnitude , 2003, Journal of Cognitive Neuroscience.

[49]  I. Weiner The "two-headed" latent inhibition model of schizophrenia: modeling positive and negative symptoms and their treatment , 2003, Psychopharmacology.

[50]  Mark D'Esposito,et al.  Rapid Prefrontal-Hippocampal Habituation to Novel Events , 2004, The Journal of Neuroscience.

[51]  Jonathan D. Cohen,et al.  Computational roles for dopamine in behavioural control , 2004, Nature.

[52]  R. Clark,et al.  The medial temporal lobe. , 2004, Annual review of neuroscience.

[53]  Timothy Edward John Behrens,et al.  Response-Selection-Related Parietal Activation during Number Comparison , 2004, Journal of Cognitive Neuroscience.

[54]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[55]  H. Heinze,et al.  Reward-Related fMRI Activation of Dopaminergic Midbrain Is Associated with Enhanced Hippocampus- Dependent Long-Term Memory Formation , 2005, Neuron.

[56]  M. Hasselmo What is the function of hippocampal theta rhythm?—Linking behavioral data to phasic properties of field potential and unit recording data , 2005, Hippocampus.

[57]  Nelson Cowan,et al.  Just lying there, remembering: Improving recall of prose in amnesic patients with mild cognitive impairment by minimising interference , 2005, Memory.

[58]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[59]  Avishai Henik,et al.  Are numbers special? The comparison systems of the human brain investigated by fMRI , 2005, Neuropsychologia.

[60]  K. Norman,et al.  Memory Strength and Repetition Suppression: Multimodal Imaging of Medial Temporal Cortical Contributions to Recognition , 2005, Neuron.

[61]  Dirk J. Heslenfeld,et al.  Activity in human reward-sensitive brain areas is strongly context dependent , 2005, NeuroImage.

[62]  Hans-Jochen Heinze,et al.  Redefining implicit and explicit memory: the functional neuroanatomy of priming, remembering, and control of retrieval. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Jeffrey C. Cooper,et al.  Functional magnetic resonance imaging of reward prediction , 2005, Current opinion in neurology.

[64]  Wolfram Schultz,et al.  Relative reward processing in primate striatum , 2005, Experimental Brain Research.

[65]  M. Rugg,et al.  Separating the Brain Regions Involved in Recollection and Familiarity in Recognition Memory , 2005, The Journal of Neuroscience.

[66]  E. Rolls,et al.  Reward-Spatial View Representations and Learning in the Primate Hippocampus , 2005, The Journal of Neuroscience.

[67]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[68]  Richard G M Morris,et al.  Dopaminergic modulation of the persistence of one-trial hippocampus-dependent memory. , 2006, Learning & memory.

[69]  Nikolaus Weiskopf,et al.  Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: A whole-brain analysis at 3 T and 1.5 T , 2006, NeuroImage.

[70]  N. Bunzeck,et al.  Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA , 2006, Neuron.

[71]  Brian Knutson,et al.  Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation , 2006, Neuron.

[72]  R. Morris,et al.  Elements of a neurobiological theory of hippocampal function: the role of synaptic plasticity, synaptic tagging and schemas , 2006, The European journal of neuroscience.

[73]  W. Schultz Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[74]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[75]  J. Wallis Orbitofrontal cortex and its contribution to decision-making. , 2007, Annual review of neuroscience.

[76]  Raymond J. Dolan,et al.  Anticipation of novelty recruits reward system and hippocampus while promoting recollection , 2007, NeuroImage.

[77]  Kathleen B McDermott,et al.  The testing effect in recognition memory: a dual process account. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[78]  Masataka Watanabe,et al.  Integration of Cognitive and Motivational Information in the Primate Lateral Prefrontal Cortex , 2007, Annals of the New York Academy of Sciences.

[79]  S. Ikemoto Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens–olfactory tubercle complex , 2007, Brain Research Reviews.

[80]  H. Eichenbaum,et al.  The medial temporal lobe and recognition memory. , 2007, Annual review of neuroscience.

[81]  Hans-Jochen Heinze,et al.  Mesolimbic novelty processing in older adults. , 2007, Cerebral cortex.

[82]  Prediction of relative and absolute time of reward in monkey prefrontal neurons , 2007, Neuroreport.

[83]  Phillip J. Holcomb,et al.  To Ignore or Explore: Top-Down Modulation of Novelty Processing , 2008, J. Cogn. Neurosci..

[84]  D. Shohamy,et al.  Integrating Memories in the Human Brain: Hippocampal-Midbrain Encoding of Overlapping Events , 2008, Neuron.

[85]  Dharshan Kumaran,et al.  The Hippocampus and Dopaminergic Midbrain: Old Couple, New Insights , 2008, Neuron.

[86]  Roi Cohen Kadosh,et al.  Are numbers special? An overview of chronometric, neuroimaging, developmental and comparative studies of magnitude representation , 2008, Progress in Neurobiology.

[87]  L. Tugan Muftuler,et al.  Multiple repetitions reveal functionally and anatomically distinct patterns of hippocampal activity during continuous recognition memory , 2008, Hippocampus.

[88]  Hans-Jochen Heinze,et al.  Novel Scenes Improve Recollection and Recall of Words , 2008, Journal of Cognitive Neuroscience.

[89]  R. Elliott,et al.  Medial orbitofrontal cortex codes relative rather than absolute value of financial rewards in humans , 2008, The European journal of neuroscience.

[90]  Howard Eichenbaum,et al.  Recognition memory: opposite effects of hippocampal damage on recollection and familiarity , 2008, Nature Neuroscience.

[91]  Jeffrey D. Karpicke,et al.  The Critical Importance of Retrieval for Learning , 2008, Science.

[92]  Omar J. Ahmed,et al.  The hippocampal rate code: anatomy, physiology and theory , 2009, Trends in Neurosciences.

[93]  E. Düzel,et al.  Personality Traits Are Differentially Associated with Patterns of Reward and Novelty Processing in the Human Substantia Nigra/Ventral Tegmental Area , 2009, Biological Psychiatry.

[94]  Nico Bunzeck,et al.  Reward Motivation Accelerates the Onset of Neural Novelty Signals in Humans to 85 Milliseconds , 2009, Current Biology.

[95]  P. Tobler,et al.  Functional imaging of the human dopaminergic midbrain , 2009, Trends in Neurosciences.

[96]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[97]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .