The Hippocampal-VTA Loop: The Role of Novelty and Motivation in Controlling the Entry of Information into Long-Term Memory

The role of dopamine has been strongly implicated in reward processes, but recent work shows an additional role as a signal that promotes the stable incorporation of novel information into long-term hippocampal memory. Indeed, dopamine neurons, in addition to being activated by reward, can be activated by novelty in the absence of reward. The computation of novelty is thought to occur in the hippocampus and is carried to the dopamine cells of the VTA through a polysynaptic pathway. Although a picture of novelty-dependent processes in the VTA and hippocampus is beginning to emerge, many aspects of the process remain unclear. Here, we will consider several issues: (1) What is relationship of novelty signals coming to the VTA from the superior colliculus, as compared to those that come from the hippocampus? (2) Can dopamine released by a reward enhance the learning of novel information? (3) Is there an interaction between motivational signals and hippocampal novelty signals? (4) What are the properties of the axons that generate dopamine release in the hippocampus? We close with a discussion of some of the outstanding open issues in this field.

[1]  P. Goldman-Rakic,et al.  The role of D1-dopamine receptor in working memory: local injections of dopamine antagonists into the prefrontal cortex of rhesus monkeys performing an oculomotor delayed-response task. , 1994, Journal of neurophysiology.

[2]  Jonathan R. Whitlock,et al.  Learning Induces Long-Term Potentiation in the Hippocampus , 2006, Science.

[3]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[4]  M. VanElzakker,et al.  Environmental novelty is associated with a selective increase in Fos expression in the output elements of the hippocampal formation and the perirhinal cortex. , 2008, Learning & memory.

[5]  Hans-Jochen Heinze,et al.  Novel Scenes Improve Recollection and Recall of Words , 2008, Journal of Cognitive Neuroscience.

[6]  U. Frey,et al.  Synaptic tagging: implications for late maintenance of hippocampal long-term potentiation , 1998, Trends in Neurosciences.

[7]  Oliver Speck,et al.  Midbrain fMRI: Applications, Limitations and Challenges , 2015 .

[8]  Ethan S. Bromberg-Martin,et al.  Distinct Tonic and Phasic Anticipatory Activity in Lateral Habenula and Dopamine Neurons , 2010, Neuron.

[9]  O. Johansson,et al.  Dopamine Nerve Terminals in the Rat Limbic Cortex: Aspects of the Dopamine Hypothesis of Schizophrenia , 1974, Science.

[10]  Hervé Simon,et al.  Efferents and afferents of the ventral tegmental-A10 region studied after local injection of [3H]leucine and horseradish peroxidase , 1979, Brain Research.

[11]  M. Guitart-Masip,et al.  NOvelty-related Motivation of Anticipation and exploration by Dopamine (NOMAD): Implications for healthy aging , 2010, Neuroscience & Biobehavioral Reviews.

[12]  N. Lemon,et al.  Dopamine D1/D5 Receptors Gate the Acquisition of Novel Information through Hippocampal Long-Term Potentiation and Long-Term Depression , 2006, The Journal of Neuroscience.

[13]  H. Happe,et al.  Localization and quantification of the dopamine transporter: comparison of [3H]WIN 35,428 and [125I]RTI-55 , 1995, Brain Research.

[14]  S. Sesack,et al.  Immunolocalization of the cocaine‐ and antidepressant‐sensitive l‐norepinephrine transporter , 2000, The Journal of comparative neurology.

[15]  Herve Simon,et al.  Origin of dopaminergic innervation of the rat hippocampal formation , 1980, Neuroscience Letters.

[16]  B. McNaughton,et al.  Differential modulation of CA1 and dentate gyrus interneurons during exploration of novel environments. , 2004, Journal of neurophysiology.

[17]  Nico Bunzeck,et al.  Reward Motivation Accelerates the Onset of Neural Novelty Signals in Humans to 85 Milliseconds , 2009, Current Biology.

[18]  M. Geffard,et al.  Ultrastructural immunocytochemical study of the dopaminergic innervation of the rat lateral septum with anti-dopamine antibodies , 1984, Neuroscience.

[19]  H. Heinze,et al.  The Dopaminergic Midbrain Participates in Human Episodic Memory Formation: Evidence from Genetic Imaging , 2006, The Journal of Neuroscience.

[20]  U. Frey,et al.  The effect of dopaminergic D1 receptor blockade during tetanization on the expression of long-term potentiation in the rat CA1 region in vitro , 1991, Neuroscience Letters.

[21]  C. Nicholson,et al.  Dopamine-mediated volume transmission in midbrain is regulated by distinct extracellular geometry and uptake. , 2001, Journal of neurophysiology.

[22]  W. Schultz.  Behavioral dopamine signals , 2007, Trends in Neurosciences.

[23]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[25]  A. Gasbarri,et al.  Organization of the projections from the ventral tegmental area of Tsai to the hippocampal formation in the rat. , 1991, Journal fur Hirnforschung.

[26]  R. Oades,et al.  Ventral tegmental (A10) system: neurobiology. 1. Anatomy and connectivity , 1987, Brain Research Reviews.

[27]  L. Swanson,et al.  The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat , 1982, Brain Research Bulletin.

[28]  B. K. Hartman,et al.  The central adrenergic system. An immunofluorescence study of the location of cell bodies and their efferent connections in the rat utilizing dopamine‐B‐hydroxylase as a marker , 1975, The Journal of comparative neurology.

[29]  Mark G. Packard,et al.  The dopaminergic mesencephalic projections to the hippocampal formation in the rat , 1997, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[30]  I. Izquierdo,et al.  Dopamine Controls Persistence of Long-Term Memory Storage , 2009, Science.

[31]  M. Geffard,et al.  Antibodies to Dopamine: Radioimmunological Study of Specificity in Relation to Immunocytochemistry , 1984, Journal of neurochemistry.

[32]  Menek Goldstein,et al.  Activation of the locus coeruleus induced by selective stimulation of the ventral tegmental area , 1986, Brain Research.

[33]  T. Dennis,et al.  Increase in dopamine and DOPAC levels in noradrenergic terminals after electrical stimulation of the ascending noradrenergic pathways , 1984, Brain Research.

[34]  C. Verney,et al.  Mesolimbic dopaminergic neurons innervating the hippocampal formation in the rat: a combined retrograde tracing and immunohistochemical study , 1994, Brain Research.

[35]  J. Lisman,et al.  D1/D5 Dopamine Receptor Activation Increases the Magnitude of Early Long-Term Potentiation at CA1 Hippocampal Synapses , 1996, The Journal of Neuroscience.

[36]  E. Düzel,et al.  Personality Traits Are Differentially Associated with Patterns of Reward and Novelty Processing in the Human Substantia Nigra/Ventral Tegmental Area , 2009, Biological Psychiatry.

[37]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[38]  S. Sajikumar,et al.  Synergistic requirements for the induction of dopaminergic D1/D5-receptor-mediated LTP in hippocampal slices of rat CA1 in vitro , 2007, Neuropharmacology.

[39]  Randy D. Blakely,et al.  Expression cloning of a cocaine-and antidepressant-sensitive human noradrenaline transporter , 1991, Nature.

[40]  G. Baldassarre,et al.  Functions and Mechanisms of Intrinsic Motivations The Knowledge Versus Competence Distinction , 2012 .

[41]  T. Hökfelt,et al.  The origin of the dopamine nerve terminals in limbic and frontal cortex. Evidence for meso-cortico dopamine neurons. , 1974, Brain research.

[42]  Daniel Paredes,et al.  Neuregulin-1 regulates LTP at CA1 hippocampal synapses through activation of dopamine D4 receptors , 2008, Proceedings of the National Academy of Sciences.

[43]  J. O’Neill,et al.  Place-selective firing contributes to the reverse-order reactivation of CA1 pyramidal cells during sharp waves in open-field exploration , 2007, The European journal of neuroscience.

[44]  J. Wagner,et al.  Dopamine transporter blockade increases LTP in the CA1 region of the rat hippocampus via activation of the D3 dopamine receptor. , 2006, Learning & memory.

[45]  L. Frank,et al.  New Experiences Enhance Coordinated Neural Activity in the Hippocampus , 2008, Neuron.

[46]  Vincenzo Perciavalle,et al.  The projections of the retrorubral field A8 to the hippocampal formation in the rat , 1996, Experimental Brain Research.

[47]  E. Kandel,et al.  D1/D5 receptor agonists induce a protein synthesis-dependent late potentiation in the CA1 region of the hippocampus. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[48]  R. Morris,et al.  Relevance of synaptic tagging and capture to the persistence of long-term potentiation and everyday spatial memory , 2010, Proceedings of the National Academy of Sciences.

[49]  P. Goldman-Rakic,et al.  Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[50]  M. Fyhn,et al.  Hippocampal Neurons Responding to First-Time Dislocation of a Target Object , 2002, Neuron.

[51]  Raymond J. Dolan,et al.  Anticipation of novelty recruits reward system and hippocampus while promoting recollection , 2007, NeuroImage.

[52]  D. Shohamy,et al.  Integrating Memories in the Human Brain: Hippocampal-Midbrain Encoding of Overlapping Events , 2008, Neuron.

[53]  R. Dolan,et al.  Contextual Novelty Changes Reward Representations in the Striatum , 2010, The Journal of Neuroscience.

[54]  Y. Agid,et al.  Dopamine deficiency in the cerebral cortex in Parkinson disease , 1982, Neurology.

[55]  W. Schultz.  Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[56]  Ethan S. Bromberg-Martin,et al.  Multiple Timescales of Memory in Lateral Habenula and Dopamine Neurons , 2010, Neuron.

[57]  T. Martin McGinnity,et al.  Novelty Detection as an Intrinsic Motivation for Cumulative Learning Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[58]  Luigi F. Agnati,et al.  The emergence of the volume transmission concept 1 Published on the World Wide Web on 12 January 1998. 1 , 1998, Brain Research Reviews.

[59]  Y. Agid,et al.  Reduction of cortical dopamine, noradrenaline, serotonin and their metabolites in Parkinson's disease , 1983, Brain Research.

[60]  M. Geffard,et al.  Immunocytochemical localization of dopamine in the prefrontal cortex of the rat at the light and electron microscopical level , 1987, Neuroscience.

[61]  M. Geffard,et al.  Antisera against catecholamines: specificity studies and physicochemical data for anti-dopamine and anti-p-tyramine antibodies. , 1984, Molecular immunology.

[62]  B. Berger,et al.  Morphological evidence for a dopaminergic terminal field in the hippocampal formation of young and adult rat , 1985, Neuroscience.

[63]  Marco Mirolli,et al.  Functions and Mechanisms of Intrinsic Motivations , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[64]  Brian Knutson,et al.  Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation , 2006, Neuron.

[65]  R. Morris,et al.  Making memories last: the synaptic tagging and capture hypothesis , 2010, Nature Reviews Neuroscience.

[66]  L. Descarries,et al.  Distribution and Morphological Characteristics of Dopamine‐Immunoreactive Neurons in the Midbrain of the Squirrel Monkey (Saimiri sciureus) , 1988, The Journal of comparative neurology.

[67]  Mark G. Packard,et al.  Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat , 1994, Brain Research Bulletin.

[68]  J. Bolam,et al.  Activity of Neurochemically Heterogeneous Dopaminergic Neurons in the Substantia Nigra during Spontaneous and Driven Changes in Brain State , 2009, The Journal of Neuroscience.

[69]  L. Quintin,et al.  Variations in 3,4-dihydroxyphenylacetic acid concentration are correlated to single cell firing changes in the rat locus coeruleus , 1986, Neuroscience.

[70]  Alcino J. Silva,et al.  Autophosphorylation at Thr286 of the alpha calcium-calmodulin kinase II in LTP and learning. , 1998, Science.

[71]  Frank Neugebauer,et al.  Modulation of extracellular monoamine transmitter concentrations in the hippocampus after weak and strong tetanization of the perforant path in freely moving rats , 2009, Brain Research.

[72]  R. Wise,et al.  Novelty‐evoked elevations of nucleus accumbens dopamine: dependence on impulse flow from the ventral subiculum and glutamatergic neurotransmission in the ventral tegmental area , 2001, The European journal of neuroscience.

[73]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[74]  H. Heinze,et al.  Reward-related fMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation , 2005 .

[75]  R. Morris,et al.  Dopamine and Memory: Modulation of the Persistence of Memory for Novel Hippocampal NMDA Receptor-Dependent Paired Associates , 2010, The Journal of Neuroscience.

[76]  S. Mizumori,et al.  Ventral tegmental area and substantia nigra neural correlates of spatial learning. , 2011, Learning & memory.

[77]  Robert E Hampson,et al.  Differential but Complementary Mnemonic Functions of the Hippocampus and Subiculum , 2004, Neuron.

[78]  F. Ballarini,et al.  Behavioral tagging is a general mechanism of long-term memory formation , 2009, Proceedings of the National Academy of Sciences.

[79]  W. Cowan,et al.  A study of subcortical afferents to the hippocampal formation in the rat , 1979, Neuroscience.

[80]  U. Frey,et al.  Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices , 1990, Brain Research.

[81]  James L. McGaugh,et al.  Evidence for dopamine as a transmitter in dorsal hippocampus , 1982, Brain Research.

[82]  David M. Santucci,et al.  A Biologically Plausible Transform for Visual Recognition that is Invariant to Translation, Scale, and Rotation , 2011, Front. Comput. Neurosci..

[83]  Ruth M. Krebs,et al.  Novelty increases the mesolimbic functional connectivity of the substantia nigra/ventral tegmental area (SN/VTA) during reward anticipation: Evidence from high-resolution fMRI , 2011, NeuroImage.

[84]  R. Greene,et al.  CNS Dopamine Transmission Mediated by Noradrenergic Innervation , 2012, The Journal of Neuroscience.

[85]  Mattias P. Karlsson,et al.  Network Dynamics Underlying the Formation of Sparse, Informative Representations in the Hippocampus , 2008, The Journal of Neuroscience.

[86]  N. Bunzeck,et al.  Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA , 2006, Neuron.

[87]  Hans-Jochen Heinze,et al.  Mesolimbic novelty processing in older adults. , 2007, Cerebral cortex.

[88]  Maria Lindskog,et al.  Dopamine in the hippocampus is cleared by the norepinephrine transporter. , 2011, The international journal of neuropsychopharmacology.

[89]  G. Flore,et al.  Evidence for co-release of noradrenaline and dopamine from noradrenergic neurons in the cerebral cortex , 2001, Molecular Psychiatry.

[90]  G. Gessa,et al.  Co-release of noradrenaline and dopamine in the cerebral cortex elicited by single train and repeated train stimulation of the locus coeruleus , 2005, BMC Neuroscience.

[91]  Y. Agid,et al.  Labeled wheat germ agglutinin (WGA) as a new, highly sensitive retrograde tracer in the rat brain hippocampal system , 1978, Brain Research.

[92]  Michel Geffard,et al.  First demonstration of highly specific and sensitive antibodies against dopamine , 1984, Brain Research.

[93]  K Fuxe,et al.  Pharmaco-histochemical evidence of the existence of dopamine nerve terminals in the limbic cortex. , 1974, European journal of pharmacology.

[94]  Emrah Duzel,et al.  A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP , 2011, Trends in Neurosciences.

[95]  W. K. Cullen,et al.  Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty , 2003, Nature Neuroscience.

[96]  T. Dennis,et al.  The formation of deaminated metabolites of dopamine in the locus coeruleus depends upon noradrenergic neuronal activity , 1985, Brain Research.

[97]  R. Wise,et al.  Linking Context with Reward: A Functional Circuit from Hippocampal CA3 to Ventral Tegmental Area , 2011, Science.

[98]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.